Shanidar

Titular notícies
Nombre de resultats 25 per a Programming

09/03/2013 - Extensibility in the Acme text editor
Text editors. You hate them or love them. Praise them with religious zeal, and attack them with the same power. I've been an emacs user for the last 8 years, getting as deep as I could without checking the source. And the past few months I have started using evil-mode in emacs, to get some taste of vim in my daily editing (mostly text objects.)

There's still a third contestant in editor-land, for me. It is Acme, the odd editor from Plan9 from Outer Space, the even-more-odd operating system from Bell Labs. There's no need to install Plan9 and fight against your current hardware. If you are in any kind of Unix derivative (Mac OS, Linux) you can install Plan9 from User Space, a port of most of Plan9 to work in user space (as you may guess.) Plan9 is a whole different thing from other Unix systems, and Acme is an incredibly different beast from any other editor you know.

I can start with a screenshot of it:

This is how this post looks like. You can see an adict window by the side
This is Acme. I hope you like this shade of yellow and this shade of blue. There's no way to change it without getting into the source code and recompiling. It may be sound odd, but I kind of like it. It's refreshing. In emacs and vim it is very easy to get a beautiful colour scheme (I use solarized-dark everywhere I can,) but this means you can choose. And choosing means a decision, with pros, cons and whatever. Just screw it and pick blue and yellow.

Once you are used to it, you have to face something "worse." If you come from emacs or vim this will sound just horrible. Wait for it. Everything is done with the mouse. Yes, you read that well. No keyboard shortcuts (well, there are a few, I'll get into these in a short while.) Mouse clicking, moving and chording. The likes. I know this will sound stupid, a waste of time, prone to carpal tunnel syndrome. Let me go on for a while.

Emacs and vim users alike like to bash any other editor in the grounds of speed. I can refactor faster than you can, is almost the motto. Watch how fast I type, thus how fast I change code. I'm one of these, I usually don't even have to think when I'm doing "something" in emacs or vim and changing stuff. But then again, how often I'm changing stuff?

Emacs and vim make easy changing what's there. Multiple-marks, text objects, quick jumps. All this is there just to make changing stuff fast. Agree? Ok, go on. If you don't, no problem. Go on anyway.

Now the revelation: most of the time I'm writing, I'm creating new stuff, not rewriting or moving old stuff. Shocking? Watch your own coding/writing habits. Yes, I change what's in text strings (ci", in vim) and is incredibly fast. In Acme, you can double-click after the first quotation mark (or just before the last) to select everything inside a pair of delimiters (a pity it is not smart enough to understand dollar-delimiters as used by LaTeX.) But the point is not that speed. What are you changing this string for? Did you wait to think about it or you just changed it, compiled it, checked it and went back to square one?

Pause: the file servers

Acme and Plan9 follow a special philosophy: in some sense, everything is a file. And most programs (I could say all, but I'm not that into Plan9 to be sure) act as file servers. Acme is just one of these: everything you can see in an Acme session is a file. For example, this text I'm writing (you saw it in the previous screenshot) has window ID 10. So...

acme/10/       is the directory associated to this text
acme/10/addr is a file with an address position for text insertions
acme/10/body is a file with the contents of the editing window (can't overwrite)
acme/10/ctl is a "file" (socket-like) that allows you to send commands to the window
acme/10/data is a file with the data of the editing window (can overwrite) 
acme/10/errors is a file with data spat by commands executed by this window
acme/10/event is a "file" (socket-like) where you can read/write the editing session
acme/10/tag is a file holding the contents of the tag (the menu above)
acme/10/xdata is a file with the data (addr bound) of the editing window

What does this mean? It means I can write code in any language that can manipulate this text. Read this out loud: I script something, and make it work with the text I'm editing. Think about indenting, linting, type-checking, done against the working copy, not the real file. With output to a special buffer associated to the file. Extending the editor is just a matter of writing a program.

Back into Acme

Back into Acme. In Acme there's no GUI: text is the user interface. TUI. Every window is composed of two pieces: the text in the window (its "body") and a tag above it (in blue.) If you want to copy something, select it and middle-click on Snarf. Then put your cursor where you want to paste and middle-click on Paste. Done. Of course, Snarf and Paste can be anywhere. In the tag menu, in the text you are editing or even in another document. They are just words. Words that do the work. 

But the same works with shell commands. I can type date and middle-click it, to get the current date in a new buffer. Same goes with ls. Or even |md5sum to calculate the checksum of some text. Or append something to a window. For example, there's an easy way to make small queries to Wikipedia via the command line (see http://www.commandlinefu.com/commands/view/2829/query-wikipedia-via-console-over-dns) I wrote a script to do it, sitting in my path, so I can now type here <wikiCLI.sh Acme with the cursor a few lines below this... and middle click.

Result:
"Acme (\; , the peak, zenith, prime) denotes the best of something. Acme or ACME may also refer to: Acme Corporation, a fictional company in the cartoon world of Looney Tunes, ACME Detective Agency, a fictional detective agency from the Carmen Sandiego seri" "es of computer games and television shows, Acme (album), the sixth album by the Jon Spencer Blues Explosion, Acme Novelty... http://en.wikipedia.org/wiki/Acme"

On button clicking

I use a Macbook (almost 5 years old already, and still kicking.) And as you may guess, it only has one button. So, how do I manage to use middle and right clicking with ease? Well, easy. Or almost. Pressing alt while clicking simulates middle click, command while clicking simulates right clicking (in Acme, not in general.) Easy, since alt is in the middle and command just right to it. 

Problem is, chording is most awesome with a real 3-button mouse. Why? I'd rather search for wikiCLI.sh Acme (text editor) to get:

;; Truncated, retrying in TCP mode.
"Acme is a text editor and graphical shell from the Plan 9 from Bell Labs operating system, designed and implemented by Rob Pike. It can use the sam command language. The design of the interface was influenced by Oberon. It is different from other editing " "environments in that it acts as a 9P server. A distinctive element of the user interface is mouse chording... http://en.wikipedia.org/wiki/Acme_(text_editor)"

Doing so is a little more troublesome: select "Acme (text editor)" and select the command with the second button (to execute) finally click: clicking sends the last selection as argument 1 to the program. Doing so with a Mac trackpad is impossible: there's no way to simulate a left-click while middle-clicking. I also find there's a glitch here: there's no way to redirect the output of the command. It's either overwriting the selection of the argument, or goes to the +Errors window.

Selections, regexes and other furry animals

How can I select everything? :0,$ and right-click. Done. Want to replace all instances of acme for Acme? Middle click this: Edit ,s,acme,Acme,g The sam syntax is easy but... odd. The first , is to select everything, s to replace acme for Acme, g for global. Easy? Not much, but Acme is just different. And works. Edit is the command to execute an editing command, by the way.

You can also do regex searches. Like :/interface/ or :/click[i|.]/. And you can get fancy, by doing filename:<search-or-position>. For example, acmescripting:/interface/ in another window would open acmescripting in the first instance of interface. And acmescripting:20 opens it, selecting line 20. As you can see, filename is implied to be current file in case of doubt. Also, this kind of referencing works nice with most compilers and linters.

Keyboard shortcuts

There are a few keyboard shortcuts, even if Acme is mouse-centric. Press Esc, and all text written since the last click is selected. In addition to this, we have the other standards:

C-U –> Delete from cursor to start of line.
C-W –> Delete word before the cursor.
C-H –> Delete character before the cursor.
C-A –> Move cursor to start of the line.
C-E –> Move cursor to end of the line.

Nothing more, nothing less. Minimal, isn't it?

Scripting power

Finally, I want to show some scripting power of acme. I introduced the concept of the filesystem a few paragraphs ago. Now, let's see how it can be used. Let's say for example I'm an avid C programmer, and like to have code neatly indented. Well, an option is to write a script that uses the indent command line program to indent the text in the window. How? Now comes a trivial example, not written in the best way possible

#!/bin/zsh
#9indent
echo "WinId is: " $winid
echo -n "1,$" | 9p write acme/$winid/addr
echo "Selected whole contents for overwriting with 'write'"
9p read acme/$winid/body | indent -st | 9p write acme/$winid/data

First is to check which is the ID of the window. When invoking a command, its environment has it in a variable, aptly named winid. To overwrite the contents of the file, we set the addr to the whole file with the selector 1,$. We do so by piping to 9p with the command write. 9p is the middleman, allowing us to read and write files in 9P servers, like the ones acme and other Plan9 programs offer. Finally we get the body, indent it (-st is the command to use stdin in indent) and pipe it to data. Done! This indents a well-formed C file as expected.

Add this file to your path and add 9indent to your tag. Ready to indent by middle-clicking.

A slightly more complex example is to generate the output of a Markdown file. The code is as follows:

#!/bin/zsh
echo "WinId is: " $winid
format=$(9p read acme/$winid/tag)
echo "Tag is " $format
echo "Format in tag"
case $format in
    *"latex"* )
        echo "latex ouput selected"
        format="latex" ;;
    *"groff-mm"* )
        echo "groff-mm output selected" 
        format="groff-mm" ;;
    *"odf"* )
        echo "odf ouput selected" 
        format="odf" ;;
    *"html"* )
        echo "html output selected" 
        format="html" ;;
    * )
     echo "Unrecognized format, defaulting to html"
     format="html" ;;
esac
echo -n "1,$" | 9p write acme/$winid/addr
echo "Selected whole contents for overwriting with 'write'"
9p read acme/$winid/body | peg-markdown --to=$format | 9p write acme/new/body
echo "Wrote the html-markdowned version to a new buffer"
last=$(9p ls acme | sort -g | tail -n 1)
echo "Get last created buffer"
echo -n "clean" | 9p write acme/$last/ctl
echo -n "0,0" | 9p write acme/$last/addr        
echo -n "dot=addr" | 9p write acme/$last/ctl
echo -n "show" | 9p write acme/$last/ctl    
echo "Moved to beginning"

This is slightly more complex, at least on the shell side. It checks the tag for one of the accepted formats for peg-markdown and then creates the formatted output in a new window, by writing to acme/new/body. Then I want the cursor to be at the beginning of this file, not at the end (as is the default.) It was slightly tricky, but the best way was to sort in numerical order and get the last-created window (that's this tail -n 1) then to set the address at 0,0 and set the dot (selection) at address by writing at the control file. Then the command show makes the window show the selected position: 0,0. Done! Intersped among all this is a "clean" command, to make this new window to close.

Here you can see a video of these scripts in a sample use (and you'll see how I miss a middle click - execute -  for a right-click - open)



Dirty, Clean, Put

A window can be dirty or clean. It is clean when the contents and the disk file are the same. It is dirty when it is being edited. The best way to know if it is dirty is if you see "Put" in your tag menu, just beside the vertical bar. By middle-clicking Put (or Putall in the main tag) you save this file and mark it as clean. 

Also, making a window clean makes closing it quicker (middle click in Del.) Dirty windows need to be Put, or you have to Del again. 

That's all, folks (for now)

I have yet to introduce the plumber, a mechanism that allows you to open arbitrary files (using rules) from within acme. For example, I can open pdf files by right-clicking on them (i.e. some.pdf) but instead of using page (the Plan9 image viewer) I use MacOS Preview. I was forced to do so, since page can't handle all the fonts in a LaTeX generated PDF, so for me it's useless. I'll probably write how I configured the plumber in my next Acme installment.

In some sense, the plumber is like a system-wide, app-deep "open" mechanism. In Mac OS, you can "open" almost anything from the command line. If you open an URL, your default browser opens it, if you open an image, Preview handles it. Plumbing is like "open 3.0" but it is hard to manage :/

22/02/2013 - Creating New Text Objects in Evil-Mode (Vim emulation layer in emacs)

The links to Practical Vim are affiliate links to Amazon. Beware!

So... last January I was in a flight to London, preparing for an intense, 12 days course on traditional shoemaking (English hand-welted shoes, improving our knowledge at The Fancy Puffin.) And my flight read was Practical Vim. Most of my readers are already aware I'm an emacs guy, so the main question is why?

I love knowing many tools. Programming in many languages, knowing enough about any thing. I couldn't stand all these people saying Vim was superior (even though most had never touched emacs,) and since knowledge is power, I thought a little more (for me) could not hurt (mwahahahaha!) I picked Practical Vim and started reading it a few days before leaving, and finished it during the few non-shoemaking moments of that course.

When I was back to having a computer, I was torn between what I learnt about Vim and looked cool and all those things I already loved about emacs. I have MacVim a few tries, writing short pieces, writing, testing, rewriting. Clearly there was no way I could give up gnus, org-mode (even though I'm not using it that much lately) and all my complex configuration options. So... how come? Even though modal editing feels weird, text objects are cool. Being able to "say" di( and get it to delete inside a pair of parentheses is way cool. Basically this is what sold me as a great feature.

I decided to use evil-mode in emacs. Best of both worlds: I still had all my emacs tools AND text objects and some modal editing. I missed having numbered lines, although I just googled for it... And there's linum-mode (since emacs 22!!) which gives them. But I miss ex commands: being able to move any line anywhere just by issuing :source m destination is powerful. Evil-mode supports some ex commands, just not copy or move. Sigh!

Today was the first day I added evil-mode to my .emacs file, and the day started by working in a LaTeX file. In a LaTeX file, most things are enclosed in pairs of dollars, like $x=1$. These dollar delimited expressions are what TeX parses as formulas, and they appear as neat images in your target (usually PDF) file. So, most of the things I need to change, delete or move around are enclosed by dollars. So:
  • change what's inside a pair with ci$
  • delete the whole formula with da$
  • delete just the contents of the formula with di$
  • copy the formula with yi$
Useful, isn't it? Problem is, evil-mode does not have $ as a text object! Luckily it offers the tools to make new text objects... Getting it to work was harder than expected, though. I was given several choices of functions I could use to create my text object. The best option seemed to be using evil-regexp-range that allows you to give a regexp for the first delimiter and another one for the second. But all tries I have done resulted in it marking the first occurrence in the buffer. No luck. After trying repeteadly with it, I gave up and used instead evil-inner-object-range, where I had to wrestle with point, mark, excursions and the like. When I managed to get it working I just decided I would not touch it again (even if I'm pretty sure the save-excursion calls are redundant, as well as most move and mark commands...) It works, don't touch it.

(defun rb-first-dollar-exclusive (&optional arg)
(interactive)
(save-excursion
(setq rb-temp-first (re-search-backward "\\$" nil t 1))
(message "First %s" rb-temp-first))
(goto-char (+ 1 rb-temp-first)))

(defun rb-second-dollar-exclusive (&optional arg)
(interactive)
(save-excursion
(setq rb-temp-second (re-search-forward "\\$" nil t 1))
(message "Second %s" rb-temp-second))
(goto-char (- rb-temp-second 1)))

(evil-define-text-object evil-inner-dollar (count &optional beg end type)
(evil-inner-object-range count beg end type
#'rb-second-dollar-exclusive
#'rb-first-dollar-exclusive))

(define-key evil-inner-text-objects-map "$" 'evil-inner-dollar)

(defun rb-first-dollar-inclusive (&optional arg)
(interactive)
(save-excursion
(setq rb-temp-first (re-search-backward "\\$" nil t 2))
(message "First %s" rb-temp-first))
(goto-char rb-temp-first))

(defun rb-second-dollar-inclusive (&optional arg)
(interactive)
(save-excursion
(setq rb-temp-second (re-search-forward "\\$" nil t 1))
(message "Second %s" rb-temp-second))
(goto-char rb-temp-second))

(evil-define-text-object evil-outer-dollar (count &optional beg end type)
(evil-inner-object-range count beg end type
#'rb-second-dollar-inclusive
#'rb-first-dollar-inclusive))

(define-key evil-outer-text-objects-map "$" 'evil-outer-dollar)


(Blogger is killing indentation, damn) Enjoy, though!


30/09/2012 - Version Control: Started using git and github (and how to set-up a remote git server)
The Octocat, Github's logo
It's been almost 6 years since I used some kind of revision control system. Back then I wasn't sure about which I wanted to use... I settled with RCS, the father of them all. RCS was structurally very simple, with text-based (human-readable) delta files. I liked that. I had all my code and TeX files under revision control, but then I started using more than one computer and it got out of hand very quickly (using RCS or CVS in Windows was quite tricky and had user and encoding problems.) Stopped using it. After a few years I tried darcs... unconviced. Then I put some of my code in the cloud, in Google Code, so I got to use a little SVN. Not for version control, just for having my code online and tidy. Nothing else.

It was about time that I got again into source code control. For part of my work I will use mercurial, so for my side projects I will be using git, to see what else is trendy right now. Yeah, it has taken me ages to do it :D

I put my thesis under control and committed a few revised sections. So far it works wonders with the built-in emacs vc-mode (version control mode.) I even get a neat menu to check in/out or check the commit story. I have not checked yet its interaction with github with push or pull, but I guess it will work as well. If it doesn't, my emacs loving twitter pals Rami and Anti have suggested the magit package. I'm not sure about it, because it is recommended when git is your only version control system, and I also want to use mercurial. Only time will tell, though.

By the way, the first real commit I did was for my work in progress with getting some acme magic into emacs. You can check it here in github (which works very well so far... why did it take me so long?).

I also set a private repository in my own server to host my emacs org files and calendar. I followed the following steps at first...
mkdir targetrepo
cd !$
git init
cd ..
git clone --bare targetrepo targetrepo.git
mv targetrepo targetrepo.backup
git clone targetrepo.git
A somewhat convoluted way to (I think) just doing this, but I'm not sure:
mkdir targetrepo.git
git init --bare
Now you have to connect to it in the local machine (and transfer the local files,) in the desired directory
git remote add origin username@server/path-to/targetrepo.git
git push origin master
And you're set to go. I got it configured in my server, Macbook and... iPod Touch. Now I have my emacs org-based calendar wherever I go, with (manual) syncing among machines. Reinventing the wheel FTW!

01/08/2012 - Adding Notes to Emails in Gnus (in Summary View)
Sensitive information blanked
Last month I switched email clients (for the 4th time in the past year.) I've passed through gnus, Thunderbird, mail.app and settled with Sparrow. Just 2 days before Sparrow was bought by Google I left for gnus (lucky moment!) Smaller memory footprint, no harddisk hungry indexes. Works wonders. Would only be happier with a newer Macbook. Oh well.

I'm still unsure about going the hardcore way with offlineimap (dovecot, notmuch,) but I'm more than happy with the current setup. I can move, delete or archive emails with just a few keystrokes. But I'm not in inbox zero yet (won't be ever, I think.) I leave some mails in my inbox when I need to follow up later (to make sure I'm on top of things.) Problem is, when you have more than 7 or 8 emails like this you start to lose touch. Furthermore, subject lines sometimes are not very descriptive "Request" is not the best subject. When more than 4 or 5 emails are of this kind, you need to check the email to know what it is about. Losing time. No way!

After all, this is emacs. If you don't like something, or want something, you write it (well, after looking for it.) My goal? Adding notes to email messages. I googled quickly for this with no luck, checked Sacha's emacs lisp files just in case. So... Time for some emacs lisping.

You may wonder: dude, use org-capture! This would be a good solution if I looked more often my todo list. But I only look my todo list when, well, I have to do something. Somehow stay-on-top things just distract me in my todo. I'd rather have 15 emails in my inbox than 15 more tasks in my already crowded todo list. I still use org-capture with emails, but for long-term catchups or long tasks where I have to do something in the future. When I just have to make sure someone does their work, I leave the email. Works for me, somehow. Getting back to lisp-topic...

It's not my first dive into emacs lisp: I've written some helper functions to do small things (wrote a wraper to the wonderful mark-multiple package a few weeks ago and I'm already using it to mark stuff at point,) I even wrote a major mode to simplify posting to mostlymaths.net, by adding my most usual HTML or CSS idioms with just a few keystrokes. The gnusnotes.el project was harder, though.

First thing was planning how to make it work. The simplest solution: get some kind of email ID, and save it to file tied to a note. I thought for 3 minutes, no need to make it more complex than necessary. Just like a CSV file, I used comma as ID-note separator, newline as ID-note pair separator. Luckily, gnus works because it has the correct pieces. In no time I had found which function determines the current message header data, and among them, message ID (the ID used in backend operations.) Once I had that, I needed a function to find the ID in the file. It was pretty straightforward (find-file and a few search-forward commands.) Something was trickier, though...

Gnus has two main windows: Groups and group summaries. For example, in Groups I have INBOX and sent-201208 as visible folders, I can check all folders (usually hidden) with L. When I open INBOX from here, I go to the INBOX summary, where all unread and marked as important emails show (and this is where my time is usually spent.) I want my notes to appear here, in the summary view. Not inside the email, not disjoint from emails. How!?

Well, for each email a summary line is generated, according to some rules you can set. You can add any detail from the email: headers, references, from, subject, date... And there is also a whole range of custom functions. Adding the flag %u you can choose from a wide range of functions (followed by any letter): In my case, %un calls gnus-user-format-function-n. This function does the lifting: checking the list of notes, for each email.

Problem: even if my notes file is small, hitting it for each email in my inbox can be dreadful. Even worse, the hit would be for any email in any folder. What if I check some archived folder with tens of thousands of emails? Well, these considerations are important, but I just found out that opening a file-buffer for each email somehow broke something within gnus, and filled the buffer with garbage. Later rationalisation turned out these problems :D

The solution (at least the one I used) is to generate a list of cons (msgid . note) each time a group is opened or a note is added. This is done via hooks, tying the loader function to the group preparation hook. Perfect! What else? The function looks for the msgid in this list of notes, and returns the note if successfull (and nothing if unsuccessful.)

To finish, just some cosmetic tweaks: adding some prettyfying of notes via highlight-regexp tied to another hook: the word note appears in italics and with the warning face, and the note appears with the default face. I'm not sure how it will look in vanilla emacs, but with solarized-dark I enjoy the colors. Completely tweakable, though.

By the way, my custom summary line is:

(setq-default
 gnus-summary-line-format "%U%R%z %(%&user-date;  %-15,15f  %B%s%) %un\n"
 gnus-user-date-format-alist '((t . "%Y-%m-%d (%a) %H:%M")
                               gnus-thread-sort-functions '(gnus-thread-sort-by-date)
                               )
 )

This is not yet a neat package with customizable options (yet). There's even no way to remove or edit notes (except by editing the notes file) since making this work appeared harder than editing this file by hand for the time being... But I may do it some day or another, it's not rocket science. Just needs some refactoring of the note-finder. Source code below, with many messages commented for dirty debugging. Enjoy!

;;; gnusnotes.el --- 

;; Copyright (C) 2012  Ruben Berenguel <ruben@dreamattic.com>

;; Author: Ruben Berenguel <ruben@dreamattic.com>
;; Keywords: comm, mail

;; This program is free software; you can redistribute it and/or
;; modify it under the terms of the GNU General Public License
;; as published by the Free Software Foundation; either version 3
;; of the License, or (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program. If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;; Still in beta. Use it at your own risk. Customize the note file
;; location and to install

;; (require 'gnusnotes)

;; Only tested in emacs 24.1.1 with NoGnus 0.18

;;; Code:


(provide 'gnusnotes)
;;; gnusnotes.el ends here


(defun gnus-user-format-function-n (data)
  (or (rberenguel/get-gnus-notes (aref data 0)) "")
)

(setq gnus-summary-notes "NOTE:")

(defun rberenguel/gnus-summary-notes-hl ()
  (interactive)
  (hi-lock-mode -1)
  (hi-lock-mode 1)
  ;(message "Highlighting %s" gnus-summary-notes)
  (highlight-regexp gnus-summary-notes 'warning)
  (highlight-regexp gnus-summary-notes 'italic)
  (highlight-regexp (concat gnus-summary-notes "\.\*") 'default)
)

(setq gnus-summary-prepare-hook 'rberenguel/gnus-summary-notes-hl)
(setq gnus-summary-hook 'rberenguel/gnus-summary-notes-hl)

;(add-hook 'gnus-summary-generate-hook 'rberenguel/gnus-summary-notes-hl)

(setq rberenguel/gnus-notes-file "~/.gnusnotes")

(setq gnus-select-group-hook
      '(lambda ()
         ;; First of all, sort by date.
         (setq rberenguel/gnus-notes-list (rberenguel/load-gnus-notes))))

(defun rberenguel/load-gnus-notes ()
  (interactive)
  (set-buffer (find-file rberenguel/gnus-notes-file))
  (goto-char 1)
  (setq moreLines t)
  (setq rberenguel/gnus-notes-list ())
  (while moreLines
    (beginning-of-line)        
    (setq p1 (point))
    (if (search-forward "," nil t)
        (progn (setq p2 (- (point) 1))
               (setq msgid (buffer-substring-no-properties p1 p2))
               ;(message "%s" msgid)
               (end-of-line)
               (setq p1 (point))
               (setq note (buffer-substring-no-properties (+ 1 p2) p1))
               ;(message "%s" note)
               (setq rberenguel/gnus-notes-list (append rberenguel/gnus-notes-list (list (cons msgid note))))
               ;(message "%s" rberenguel/gnus-notes-list)
          )
      
      )
    (setq moreLines (= 0 (forward-line 1)))
    )
  (kill-buffer)
  rberenguel/gnus-notes-list
  )

(defun rberenguel/get-gnus-notes (msgid)
  ;(message "Get notes: %s" msgid)
  (setq gnus-note "")
  (unless (null rberenguel/gnus-notes-list)
    (dolist (item rberenguel/gnus-notes-list)
      ;(message "%s %s %s" item (car item) (cdr item))
      (if (equal (car item) (format "%s" msgid))
          (progn
            (setq gnus-note (format "\t NOTE: %s" (cdr item)))
            (return))
        ))
    gnus-note
    ))
      
(defun rberenguel/gnus-notes ()
  (interactive)
  ;(message "%s" (car (gnus-summary-work-articles 1)))
  (setq rberenguel/gnus-notes-msgid (car (gnus-summary-work-articles 1)))
  (setq rberenguel/got-message (rberenguel/get-gnus-notes rberenguel/gnus-notes-msgid))
  ;(message "Got Message? %s"  rberenguel/got-message)
  (if (or (equal rberenguel/got-message nil) (equal rberenguel/got-message ""))
      (progn 
        (set-buffer (find-file-noselect rberenguel/gnus-notes-file))
        (setq gnus-new-note (read-from-minibuffer "Enter Note: "))
        (end-of-buffer)
        (newline-and-indent)
        (insert (concat (format "%s" rberenguel/gnus-notes-msgid) "," gnus-new-note))
        (save-buffer)
        (kill-buffer)
        (rberenguel/load-gnus-notes)
        (gnus-summary-prepare)
        ;(gnus-summary-rescan-group)
    )))

23/07/2012 - Juggling for Programmers (and other computer people)
From Flickr
No, this is not a tutorial (but I do have written the standard steps to learn below). I began learning to juggle very recently. I just want to answer a possible "why"?

I've always been fascinated by juggling, and every time I wanted to eat more than 2 small oranges I'd try my hand at juggling them. Always followed by bowing and picking them from the ground, of course.

A few weeks ago I was in Dublin for a few days, and in a bookstore I saw a pack of juggling balls (3 balls) and promptly bought them. With them, a small leaflet explained the method, and as soon as I was back home I tried to learn the trick. I'll give you a quick overview:
  • Even if I know it sounds lame, start with one ball, from hand to hand. You have to get a knack for the height and arch. The throw is not up, but up and to the direction of your other hand. Every time you throw, think "throw".
  • When you are confortable with one-handed throws, pick a ball in each hand. Throw your first as before, and as soon as the ball reaches its maximum height, throw your other ball as before (and think "throw" as we did before). Repeat this an awful lot. Get the hang for the timing, height and getting used to thinking "throw".
  • After a while, you will want to throw three balls. Okay. Start with 2 balls in your weak hand (left if you are right handed and right if you are left handed)). Start by throwing one from the left and then one from the right just you did in the previous step. Now, when the first ball lands in your right hand, your second ball should be in the topmost position. Now is the moment to throw the next ball! Don't worry if it falls. Get used to throwing it. Thinking (or saying aloud
  • Rinse and repeat... And then get to throw your fourth ball from the right hand. Then your fifth...



But what's the point? Of course, it's fun to watch, and maybe it is fun to do for a while. But, in the long term?



I can't be sure, I'm just a data point and I've been in it for just shy of 3 weeks (on and off, since in July I'm doing quite a lot of The Fancy Puffin work and it cuts my "fun" time.) But this is what I found.

First thing I noticed, juggling clears my mind pretty effectively. It is not as good as a good meditation session, but while you are juggling (or at least, in the beginning) you can't think of much else. I practised for a few minutes each night

After a few days I noticed something else. My back felt straighter, and I had less back and shoulder pain. When you are juggling you have to keep your back very straight, stand up and move your arms. Something most programmers and people working with computers don't.

I don't know if both these secondary effects will last long. But I thought some other fellow geeks would be interested.

12/06/2012 - Timeline of whatlanguageis.com: first Django project
Last week I wrote about whatlanguageis.com, my first simple try at creating a Django powered site. How did it came to be?

In the last few months have been writing and thinking about several ideas to process social data with Python, undoubtedly motivated by the great book Mining the Social Web. In the back of my mind, I wanted my code to work as a web application (either for me or for others,) and thought that I should comply with everyone else, learn PHP and try to make Python and PHP be friends. Of course, this is like getting a Eskimo to talk to a Polynesian. It may work, but needs a lot of set-up from both sides.
This suggests the next question: why me, PHP? Well, where I work the de facto language for web development is PHP. I'm not a developer there, I just do the data analytics and whatever else that needs to be done using the tools I need and just report back. I thought that PHP-ing would help me get it in the workflow. But, even if my mother programming language may be C (although my first words were in GW-Basic,) my languages of choice are Python and Lisp (and I have a great fondness for Forth.) I guess you realise how ugly I feel PHP is.

On 29th May I decided enough was enough: if it is something that should almost work as a black box, to automate the part I was already doing with Python code and R data churning, why bother? Just make it work. I started reading the free, online Django Book, and I was hooked (you can buy a printed copy in Amazon). It is an awesomely readable book. I had just finished reading Zed Shaw's Learn Python the Hard Way to find things I may not know yet (for example, unit testing... silly me) and Django felt just like an extension of what I usually write and what I had read. Views, urls, models, all have this Python feeling. For me, of course, I'm not a seasoned Python developer at all. I don't even consider myself as seasoned in any programming language, I just know I can write code as I need. But no expert.

I read the first 2 chapters very quickly, and on Wednesday 30 I started testing how Django worked, basic view-url combinations and a test on templating. Just the hello world tests, so to say. I set aside the book while I was thinking what would made a good initial project.

Related to my social media endavours I had a tickling question: How do I decide if a twitter user data is valid or not? I.e. if I want to do some kind of language analysis, I need to know the user 's language. I had a simple idea around that week (I think it was Monday 28,) but didn't get to code it until the 4th of June (Monday): you can read about it in Language detection in Python with NLTK and Stopwords.

On Tuesday 5, after work I created the Django project whatlanguageis in my computer, and read the chapter on Forms. Created a very basic stripped HTML (I think I didn't even add a head section...) page where I could enter some text and get a language as a reply from the Django test server. Yay! It was very easy to do: Django does a lot of heavy lifting and this was straightforward to code.

Then on Wednesday 6 I took the day slightly off (only answering to urgent emails and sales enquiries) and wrote the whole site whatlanguageis.com, which involved:
  • Reading the chapter on Models to add a little database of affiliate books, checking how it works
  • Reading the chapter on the Admin interface to add some books easily, checking it
  • Reading the chapter on Deployment...
  • ... which involved a lot of VirtualHost tinkering with my httpd.conf file
  • Purchasing a domain and setting up the DNS records
  • Designing the page (this was the quickest part, Twitter Bootstrap to the rescue)
My initial plan for Wednesday was not of doing everything and not working. My original plan was just to hook the database and check I could read and write the data as expected, forcing me to read the Models chapter. But as soon as I had finished this hooking and checking, the site was almost ready, needing only a neat dressing. Some twitter bootstrap and css-arrows later, the site was working wonderfully in my local machine. Getting it to work in my remote machine was trickier, due to the setup of static files in Django. It's not hard, it is just that it is tricky to find where it fails when it does. In any case, I got it working.

Of course, as I have said a lot of times, whatlanguageis.com currently is just a proof of concept of two things: language detection and setting up a Django site. Both worked very well, but now I have to sit down and refactor the whole of it. 


The site works well as it is (no need of much refactoring except for the needed changes in views), but the language detector needs a proper module, unit tests and a huge revamping. Among others, my future plans for whatlanguageis.com include:
  • Adding a lot of languages. More or less I will reuse the ideas from an old post: The 100 most common words in Icelandic, automatically generated from Wikipedia
  • Add the ability to also detect programming languages (or at least, the most common)
  • Probably add the ability to detect language families (i.e. when unsure, classify as Scandinavian, or classify as Lisp-like)
  • Improve the results: Unless the analyzer is completely sure I want to know what it could be (i.e. for very close languages like Norwegian Bokmål and Danish detection is hard, thanks to @qimarc for this specific and useful example)
  • Add checks for word diversity in the received text
  • Unit-test all the things!


I hope you enjoyed whatlanguageis.com tale and liked the memes lying around. Somehow I felt this post needed them. Thanks to memegenerator.net for the ability to create them :)


26/01/2012 - Learning to use vim in my iPad (even if I'm an emacs geek)

Just in case you don't know, vi is an advanced text editor, drting back from the same era as emacs was developed (emacs started slightly earlier). Sort of the Jekyll to emacs' Dr Hyde. Emacs users despise vi users, and vi users mock emacs users. This is what the editor wars are all about: "Eight Megabytes And Constantly Swapping" versus "vi has two modes: writing and beeping". If you have been long enough in this blog, you know I'm in the emacs side, but you also know I'm curious enough to delve into the other side.


My first contact with vi was in our first programming class at the University, Informatica (Computer Science). Vi was the editor of choice for the course, so much that you did not have any other option given. Along with a Linux cheatsheet and a short C manual came a vi cheatsheet. A lot of people came to hate vi, a lot of people came to love it. I was more in the hating side, but it wasn't that bad. After a while I just started using a "normal" Linux editor (it was Kate, maybe?) and kept using it (or UltraEdit when I was in Windows) until the end of my degree.

When I finished my degree I had to be a teacher of Calcul Numeric (Numerical Analysis), the third (and last) programming class in the degree. At home I was still using Windows, because the wifi card in my notebook had no Linux drivers and in my office we were using Ubuntu. I needed a cross-platform editor that was quick, versatile and could last for a lifetime. I tried vi (again) and emacs, emacs won the war for two reasons. First was the fact that it was extensible programming in Lisp, and I had already a decent knowledge of Lisp. The second was AucTeX and its "preview" option for LaTeX editing. Nothing can beat pressing C-p C-p C-d and seeing your formulae come to life in your editor. I became an emacs advocate for this reason, then after 5 or 6 years I've found many more reasons why this is the editor. Or, the operating system, if you prefer.

But advocacy is nice, but knowing all the players in the field is better. My department pals used vi (at least most of them), and it was impossible to convince them to switch. I decided learning vi (or vim, vi iMproved) was something I had to do some day, to learn what was there. After all, if I liked it so much I could set viper-mode in emacs and use vi keys in emacs. The best of two worlds, if that world is so nice.

Then the guys at Applidium released a Vim port for iOS devices. Whoah! Even if I'm an emacs guy to the bone, a modal editor is way better than anything else available in the app store, at least for raw editing power. And I say with knowledge, because I've tried most writing apps for iOS devices (see this post).

As an additional reason, Matt Might posted a New Year's post detailing several resolutions for programmers. Among them was breaking your comfort zone to keep your mind sharp. The first example? Switching to vi from emacs or vice-versa. He used to be an emacs user before getting to use Vi. I don't think I could go that far, but these were enough reasons: advocacy (from me, not against me), iPad+bluetooth keyboard and breaking my comfort zone.

How is the experience so far?

Well, I'm writing this post in my sofa, with my iPad and my bluetooth keyboard sitting in my lap. I'm still getting used to having to exit editing mode to move around: I'm very used to pressing C-a or C-e to go to the beginning of end of line. An additional problem is that even with the bluetooth keyboard, Esc can't be mapped to the Esc event, and I needed something to do it. I have jk (pressed together) for it. It was a suggestiond I saw in Hacker News' comment thread for the release of the app, and it's quite handy. Not so much with the on-screen keyboard, but for now it's okay. To remap this, press backslash (the current mapping for Esc) and then :imap jk . Of course if your usual writting language involves writing the jk combination frequently (I think I could come up with one or two examples in Icelandic), this is not the best combination. The other suggested option is :imap al . Easy to tap in a virtual keyboard, not so straightforward in a normal keyboard.

I found out also how to change the font size, at least for now this works (maybe there will be more fonts available in the future), type :set guifont=Courier:h24 (for 24 pixels)

To round everything, I just installed the famous solarized color theme, dark in vim for iPad. Until now I had just thought it was an overhyped color scheme: I had tried in my emacs and didn't enjoy it that much. But in my iPad it shines with a distinct colour, it's so much easier on my eyes that it hurts to get out and write an email! To install it in your vim for iOS, first download solarized.vim from the git repository, then plug your iDevice and use iTunes File Sharing to copy this file to the vim app. Then open vim, :e solarized.vim and :call mkdir(.vim/colors/) (to create the directory needed for it) :w .vim/colors/solarized.vim This is just because I didn't seem to get iTunes File Sharing to work with hidden files (a file starting with a dot is hidden). Then you have to add the following to your .vimrc:
syntax enable
set background=dark
colorscheme solarized
I recommend using this app for any emacs lover out there with an iPad. Of course, it is not emacs, but vim is a pretty awesome text editor, and it's always handy to learn to use it.

To have a useful list of vim commands at hand I installed the Vimmy app, a universal app with the most common vi commands. I can switch to it via the multitasking gestures in my iPad, or use it in my iPod Touch while I'm using it in the iPad.

Conclusion

I don't think I'll ever switch from emacs to vim any time in the future, as I said AucTeX is definitely too good. Of course since then I've found many more things I love: having a REPL for Python, Clojure or Lisp inside my editor. There's even a REPL for PostScript! Also local remote editing with tramp... There are too many things I use on a daily basis and I can't barely remember, they are so entrenched in my .emacs file I can't even realise what they are. But for my iPad it is an awesome addition to write text on the go.

Again, if you have never used vim before and are interested, give it a try. Of course the first time you open it it will be... a jump into the unknown. Tap the screen, press :e filename and start editing happily. Esc (well, jk or backslash, as the initial mapping) :w to save. And you can even :q to exit to springboard!






11/12/2011 - Good Books I've Read in 2011: Perfect Gifts for Geeks!

Taken from Flickr
The links to books in this post are affiliate links! Beware :)
In case you have not realised it yet, I'm a pretty prolific reader. Online reading (and having an iPad) have slowed down the number of books I read in a given year, and I don't go to the lengths of my girlfriend (who is about to reach her goal of reading 102 books in this year,) I'm nevertheless a frequent reader.

This year I've read several good books that I'd like to share with you, after all, if you are reading this probably our tastes overlap. And then these books will be perfect for you (or for some geek in your life). The books I've selected range from programming, sports and non-fiction. Oddly enough, I don't remember any fiction book I've read this year (not counting some book by Raymond Chandler, which I can recommend each year). Let's go!

Born to Run A Hidden Tribe, Superathletes, and the Greatest Race the World Has Never Seen: I reviewed this book in a previous post here. I found it very good, and urged me to get running. My running frequency is (to put it softly) awful, but I'm trying to improve. If you want to start running or recover some lost motivation, give it a shot, it's a very good read. You can read my review of Born to Run.

Visualize this The FlowingData Guide to Design, Visualization, and Statistics I wanted a book to help me graphing things in my future work as a freelancer, and after looking around a little I found this *great* book by Nathan Yau, the blogger behind Flowing Data. It's an awesome book, which will help you start kicking around the graphs and tables.

Confessions of an advertising man: An almost autobiographical book by famed ad-man David Ogilvy, if you are anything into copywriting, advertising or marketing, you have to read this book. Not only will it help sharpen your ideas, but it is also a very good and entertaining read.

Moonwalking with Einstein The Art and Science of Remembering Everything: A New York Times best-seller by Joshua Foer. After reading his interview-article there, I decided to buy it (pre-ordered it!), and it was very worth the money. An entertaining read of how motivation can get you a long way, and how some memory techniques work. I have not written a review of it, but you can read my own explanation of the memory palace technique.

Even a geek can speak Low-Tech Presentation Skills for High-Tech People: The best book for anyone in need to give a presentation. Clear, concise and to the point, this should be a forced reading for all technically minded people. You can read my review of Even a geek can speak.

Mining the social web Analyzing Data from Facebook, Twitter, LinkedIn, and Other Social Media Sites: If I had to try the book I enjoyed the most reading this year, it is this book. Seriously. Even awesome comes short of what I think about this book. I plan on reviewing it, but I've got not enough time yet... With it you'll learn how to mine twitter, Facebook, LinkedIn and whatnot. And then process all this data. Easily. Yes!

The AdWeek copywriting handbook The Ultimate Guide to Writing Powerful Advertising and Marketing Copy from One of America's Top Copywriters: Your deep entry into copywriting, from a master copywriter. You'll learn how to shape your online copy, from headline to action. It's very well written in an understandable fashion. Keep in mind that if you buy the Kindle edition, the provided checklists are not that useful.

Thinking Forth: I've had this book for a long, long time and I finally got to read it. I used a simple method for getting me to read it: I left it in the bathroom. When I had to spend some time there (for example, shaving with an electric machine!) I read a few pages. And in no time, I had finished. You don't need to have a computer close to read it, but a little knowledge of Forth is quite useful. It is a clean book, introducing some concepts like orthogonality and testing without even trying.

Power Sleep The Revolutionary Program That Prepares Your Mind for Peak Performance: A best-seller book focused on mental abilities and how sleep affects it. Packed with research results and advice, it was a very interesting book, although a little too long. After all, the best suggestion is to go to sleep each day at the same time and keep adding 15 minutes (weekly) to your sleep until you stop feeling sleepy. Now that you have the spoiler, buy it.

Crush It! Why NOW Is the Time to Cash In on Your Passion: One of these self-help books that will get you pumping ideas and rocking you off the sofa. According to Gary, now it is time to crush it. And probably it is. If you are a little low on motivation, read this book. Don't expect it to teach you how to do things, just read it as experiences to think about.

What I talk about when I talk about running: Another running book, this time an autobiographical piece by one of my favourite writers, Haruki Murakami. I was expecting a little better from him... But of course, not every book can be Hard-boiled wonderland and the end of the world. An interesting read about Murakami's running experiences, and how he got to run a 50 miler and the "original" marathon in Greece (under a scorching sun).


This month I've started several books that I'm sure would have make this list, and will probably be delayed until next year:

Steve Jobs: I will remember how and when I found out Steve died, and will still be inspired by his commencement speech for a long while. I've gone through almost one fifth of the book (a very thick one,) and it is one of the best biographies I've read (the other is John Nash's). Brilliantly written, it reads like a novel with a fast pace.

1Q84: the recent best-seller by Murakami was the gift from my girlfriend for this year's St. George day, and I've only been able to read a few pages. From the raving reviews, I assume I'll like it (I was deceived by Kafka in the shore).

The memory palace of Matteo Ricci: A classic book for memory techniques lovers, I've only got to read the first 20 pages or so. So far, very interesting and well edited.

Guns, Germs and Steel The fates of human societies: A book recommended by my thesis advisor, dealing with why Europe invaded America and why it was not the other way around. Of course, all is theoretical, but it is amazing nevertheless.

If you find some broken link, please let me know!


03/12/2011 - Working on the go with an iPad, a Bluetooth keyboard and a 6sync account
All hail Steve Jobs
Some of the links in this post are affiliate links. As usual, I only recommend what I've used and like

Inspired by Mark O'Connor from Yield Thought (my frequent readers will have already read something from him from my link collections), I have been working remotely for a week. His set-up is an iPad 2, Apple wireless keyboard, the iSSH app and an account in Linode. My setup is similar, but I use an iPad 1 and 6sync for the VPS.

Why 6sync?

A matter of luck. Around 2 weeks ago, I set up my "name domain" at rberenguel.com (currently down, I messed the Apache configuration a few days ago), and I decided I had played with blogger for long enough. Decided to look up an VPS to host that site and whatever I needed to either host or cloud process in the future. Tweeted asking for options, and in the meantime, Sacha Chua explained how she switched to Linode. Linode was going to win...

Until the guys at 6sync (Mario and Rav) sent me a tweet, urging me to check 6sync before choosing. I checked the page, the prices, and was close to sold out. Exchanged a few tweets, read a few reviews, and I was convinced. Reasons? Many.

First, they offer a more basic and cheaper service. Whereas Linode offers 512 MB of RAM, 20GB storage and 200GB transfer for 19.95$/month as the basic plan, 6sync offers a nano plan with 256MB of RAM, 12GB storage and 150GB transfer for 15$. This plan is currently ideal, since my bandwith and RAM usage for the server side is close to nil, and I don't expect to use that much of bandwith in daily doses. They also offer a Mini plan with 2 CPUs, 512 MB or RAM, 20GB storage and 400GB transfer for just 20$/month, beating the Linode offer on its own ground.

Second, they have a very nice website. As a hobbyist page creator, this is important. I like how they have set-up their site, and the pictures they choose to go by. If they are not the designers, the designers should receive this praise.

Third, they are a funny bunch. I exchanged several tweets with them, and they were very supportive. And as soon as I had my server up and running and ran into problems (or more precisely as it turned out, I was not patient enough for the DNS switch) they were extremely quick to help through their in-page customer chat service.

Fourth, they have a lot of server setups ready. Do you want Debian? Check. Ubuntu? Check. Arch...? Yes! I can use me dear pacman package manager and install whatever I want, in the latest version. Cool.

All in all, very happy for hosting. And then, Mark published his post.

The remote and local setups

At first, instead of filling my 6sync server full of software, I tried a simpler setup: using my office computer in the department. I still have an account for using that computer, and I have already everything setup for working remotely. I only needed a decent ssh client for iPad.

Back in the iPod Touch days, I used the awesome (truly awesome) TouchTerm app, which is (was) perfect for the iPod Touch. Offering a wide array of gestures to control it, it is a marvel of usability. But it has no iPad version. For the occasional SSH session I was using the free zatelnet app. But this also was not a good deal.

When Mark published his post, I decided to buy iSSH, and even if the price tag is high for an app, I don't regret it. The freedom you get when only carrying an iPad and a very slim bluetooth keyboard is unmatched. The app is universal, so you have two apps for the price of one. Check the server stats from your iPhone!

Mark is a vim guy... I guess that's okay for a lot of people, but not for me. I could go the vim way (I have been trying to get a basic proficiency of vim in the past), but I'm an emacs guy. Up to the last moment. And I had to cross fingers for iSSH and emacs to play along nicely...

Well, they mostly do (at least with iOS5). The control key works, and so does the option key (as meta). The real meta key (command) does not work as expected, but esc also works as meta, in case of need. The only problem I've found so far is the fact that control-f and control-b are hard-bound (in the iOS level) to advance/go back a word. Thus control-x control-f does NOT open a document. But if you press shift, emacs does not care about control-X control-F, and the iOS bound setting is overriden.

There's a more important problem with these iOS level shortcuts, control-a goes to the beginning of the line... but leaves a nasty [] in the buffer. I have written an abbrev-expansion and re-bound control-a to get rid of it, but it does not work as well as I want yet. Luckily emacs is flexible enough to allow me to solve this in more than one way. Currently almost done anyway.

Installing all the stuff is a breeze. Of course, I had already installed the LAMP setup, although I only have configured Apache so far, but adding the rest of things I use on a daily basis is straightforward:
pacman -S emacs-nox texlive-core
And now... Dropbox or not dropbox? Dropbox from the command line has a slight problem: you need a browser to accept the linking between your new computer and the remote system. And I didn't want to add the complexity of X to the game (even if iSSH has a very cool X server...). So I left dropbox out of the equation. I can always push/pull the files every once and then without problems.

So, I have everything that I need. Now, how does it work in practice?

The daily grind

As you may already know, my daily grind is split between my part-time freelance job as social media manager (among many other things covering get more pageviews and make more money from the sites, mixed with data analysis and copywriting) and finishing my thesis.

Working in my iPad for the freelancing job was already possible. I could access almost all systems from just a browser. Reading and answering mails, the same, although going from the email app to the CRM is slow. So far, the only problems I've found are spreadsheets (solved since I got Numbers) and e-Junkie. e-Junkie has the nice feature of having a flash-based login box. Clever, isn't it? Well, I'll leave the e-Junkie management for my MacBook moments.

For the thesis part, it was harder. There is an awesome LaTeX editor for iPad, TeX Touch. I can't really stress how good it is. Using it with the on-screen keyboard is as close as you can get to be proficient. The drawback is that once you're used to emacs, if you have a solid keyboard, nothing can beat emacs except telepathy.

This is where the iSSH stuff enters into play. Fire emacs, edit all your TeX in console mode (I can somehow live without AucTeX's preview, although it is slightly hard), compile the pdf and move it to the shared folder (all done via an automated script within AucTeX for compiling). Switch to Safari (or GoodReader) and open the document. Check everything is in place and keep going.

The standard TeX based lather-rinse-repeat is straightforward from within an iPad.

Anyway, I'm not using only the iPad day to day. Only those days when I know I don't have that much work to do, requiring heavy task-switching or firing tons and tons of emails with attachments. For slow (or semi-slow) days, it is awesome.

What benefits are in this?

Well, I'm free of carrying a heavy MacBook. Until I get a nice MacBook Air, my current computer is a slightly too heavy MacBook from 2008. It weights quite a lot, and I need to add the charger (even if it is very light, it weights). When I carry the MacBook, I also carry the protecting case (more weight) and a backpack (weight) with some folders with the stuff I'm correcting for the thesis (I could remove this). And I also carry my iPad.

These days I get by with my purse-bag, which is perfectly sized for the iPad+keyboard. I carry the pages I need to check for the thesis, and no more. So I've gone from 5-odd (or 6-odd) kilograms to slightly more than 1.

I also don't need to worry about plugs or battery, the iPad battery lasts for a days' worth of working.


26/01/2011 - Installing Debian on my Ben Nanonote



Yes, that's me with a polariser.
Wait, playing Dungeon Crawl
in your Nanonote?
Installing Debian on the Ben is pretty easy after the work of the people at pyneo.org. I follow mostly their instructions which can befound here.

The first step is installing the package xburst-tools, following the instructions from the Qi Hardware Wiki about reflashing your Ben Nanonote.

The package can be downloaded in .deb format for Debian-like distributions. As I am usually in Arch Linux, I installed the confuse library with pacman -S confuse , downloaded the package tarball and uncompressed it, following the instructions therein. The only drawback is that uncompressing the tarball puts the executable files in binary directories which are not in my path, and to use usbboot (the main program talking to the Ben waiting to be reflashed) I had to use global paths. No big deal, though.

After getting this program, you'll need the boot image and the kernel, which can be found in the Qi Hardware's site as Nanonote software repositories (latest) Be sure to pick one file named (something like) openwrt-xburst-qi_lb60-u-boot.bin (the boot part), and openwrt-xburst-qi_lb60-uImage (kernel image). If you want to have sound enabled in your resulting Debian system, be sure to pick also the file openwrt-xburst-qi_lb60-rootfs.tar.gz (this may be deprecated after changes in the OpenWRT build process)

Now you have the boot and kernel, you'll need to download a Debian rootfs prepared for the Ben. You can find one for Debian-Lenny in the Pyneo.org page, get it with either
wget http://pyneo.org/downloads/nano/debian-lenny-mini.ubi
wget http://pyneo.org/downloads/nano/debian-lenny.ubi
The image I use is a debian-sid which you can get with
wget http://mister-muffin.de/nanonote/debian-sid.ubi
You could also follow the instructions in Qi Hardware wiki about debootstrapping any distribution for the Ben NanoNote, but it will be harder and messier.

Now you have all the ingredients to install Debian on your Nanonote.

Be sure to have it shut down and connect it with the supplied cable (directly to your computer, no USB hubs whatsoever) and start it in USB boot mode (by pressing U and the power button). Run lsusb (list USB devices) in your console to check for changes, when 601a:4740 appears (or something like Ingenic LB60, depending on your system), you are ready to do the following (as root or using fakeroot).

In what follows, path-to-loader is the path to the u-boot file, path-to-kernel to the uImage and path-to-rootfs the path to the ubi file with the filesystem (you should use the Debian you just got,
but could use Qi Hardware's OpenWRT image.
usbboot -c "boot"
usbboot -c "nprog 0 path-to-loader 0 0 -n"
usbboot -c "nprog 1024 path-to-kernel 0 0 -n"
usbboot -c "boot;nerase 16 1024 0 0"
usbboot -c "nprog 2048 path-to-rootfs 0 0 -n"
Don't skip step 4: I thought that the nerase part wasn't needed as I was just overwriting the filesystem, but if you don't, it won't work and you will be left wondering what is up.

Now unplug your Nanonote, remove the battery and put it again (it is the easiest way to restart it 100% sure). Boot it up and you should get a nice Debian prompt after the kernel loading screen (which will still show OpenWRT's logo).

You are done! No, not really. This barebones system needs something useful. More so, networking and sound enabled!

Getting sound working in your Ben NanoNote with Debian

To get working sound I used a microSD card, if you don't have one you will need to set up networking and some scp enabled system in your Nano.

To get sound working, uncompress the rootfs.tar.gz in your /tmp/ directory. Then copy everything from /tmp/lib/modules/kernelversion/ to your card or Nanonote. In my case, kernelversion was 2.6.32.16. In your Ben, create a directory named kernelversion in /lib/modules/ and copy everything in this newly created directory. The files you just copied are the kernel modules to get sound working.

Run depmod /lib/modules/kernelversion/*.ko This will generate (among a few others) modules.dep in /lib/modules/kernelversion/. You are ready to go! Edit /etc/modules and add snd-soc-qi-lb60 at the end of the file and issue a reboot. You should see the following when the kernel ends its boot up steps:
No device for DAI jz4740-i2s
asoc: jz-codec <-> jz4740 mapping ok
Sound should be working, although you will need some alsa magic to get it to do something. For it, we will need networking. By the way, Debian with sound enabled as a kernel module takes around 17 seconds to boot in my Nanonote.

Networking in your Ben Nanonote

Now you need networking to check if sound really works, we'll get some useful software in the Ben along the way.

I have successfully set up this in both Linux and MacOS hosts. Start by plugging your Nanonote to your host with the supplied cable. We will configure the host system first, then the Nanonote.

For Linux

You may need to be root to do this, I don't remember. First you'll need to enable network forwarding with
echo "1" > /proc/sys/net/ipv4/ip_forward
Now you set up the network interface with
ifconfig usb0 192.168.100.1 up
This is it for Linux hosts, at least this was enough for me in Arch Linux. The wiki page on Networking in your Ben NanoNote has a little more information in case this does not work.

For MacOS

When you plug your Nanonote, a network preferences alert will pop up, asking you to go to Network preferences, like this one:



There is a catch, though: it may fail to appear. To get it to appear then you have to start playing with power up power down your Nano and switching USB ports. I'm not the only one with this problem, agrajag also pointed it out.

I got my Mac to forget about previous connections by using a USB hub to plug the Nano, then unplug it and plug again in the usual spot. Rebooting the Mac also solves it.

CAUTION: I tried to use a hub to connect to internet, and after enabling internet sharing my Nano ran out of memory and ended kernel-panic-ing.

Go there and set up the newly discovered ethernet adaptor and assign an IP manually: fill in with 192.168.100.1



Now go to the System Settings pane and open Sharing. Enable sharing your internet connection with the newly discovered ethernet device and apply, like the following


Common steps in the Ben Nanonote

First set up the network interface with
ifconfig usb0 192.168.100.2 up
And enable a network gateway through your host system
route add default gw 192.168.100.1
Now point your DNS server to Google's DNS servers with (you'll only need to do this the first time you try networking)
echo "nameserver 8.8.8.8" >> /etc/resolv.conf
This should be it check with ping google.com and it should work if the host steps were done correctly.

Installing shiny new software

With the packages listed here (maybe you can add octave and some other programming lanaguages), the Ben Nanonote turns into one of the most complete (non-graphing) pocket calculators out there. In theory, you could use fim or fbi to display images generated by gnuplot to a framebuffer console... but I couldn't manage to get it to work without either seg-faulting or crashing. You can also find a list of interesting packages and hacks in Marc Zonzon's webpage.

Start by updating the system:
apt-get update
Now you are ready to fill it up. Recommended stuff (i. e. What I put no matter what). Small descriptions included.

emacs23nox The No-X emacs 23 build. It works pretty well in the Ben, even with just 32MB of RAM. Having emacs in your pocket is awesome.

zile Zile is lossy emacs. It is a quicker loading small clone of emacs. Mostly keybindings, but for quick edits it loads way faster than the full fledged emacs

fbterm Frame buffer terminal. With it, you can resize fonts with fbterm -s fontsize

crawl Dungeon Crawl Stone Soup, one of the best dungeon crawl games out there. To get it to work (albeit very slowly...) you will need an SD card ext2 formatted to get swap memory in it (more on this later)

nethack Works out of the box without needing swap hacks

yacas Yet Another Computer Algebra System. Sadly it is no longer maintained, but it is awesome what it can do in just a tiny machine like the Ben.

pari-gp The calculator for number-theoretic people. I have also ran some computations in Pari, its programming language is pretty simple and its high precision arithmetic routines are awesome

alsa- You will need all the family of alsa-utils, dev and so on. apt-cache search alsa and pick almost all.

madplay a command line music player I used in my powernap script

cron If you want to run something periodically you need cron

gcc Are you compiling something in your pocket or you are just happy to see me?

Caveat: You may see the dreaded Bus errorackage list 0% when apt-getting or updating. This usually solves by deleting /var/cache/apt/*.bin (or /var/apt/cache/*.bin) and apt-get update again. If this does not solve it, clean completely var/cache/apt (files and directories), reboot and apt-get update. This made the trick for me, if it does not for you... Well, you can try re-installing the system.

Swap

Okay, if you want to run crawl or just get short on memory, you will need either a swap partition or a swap file. As I am not into messing with partitions, I created a swap file. You will need an ext2 (or ext3 too, I think) formatted microSD card, mounted in /mnt/
dd if=/dev/zero of=/mnt/swap/swapfile bs=1k count=20480
mkswap /mnt/swap/swapfile
swapon /mnt/swap/swapfile
This will add a 20 MB swap file to your Nanonote memory. Currently I have a 50 MB one, previously I had 5 blocks of 10 MB that I swapped on when needed.

Before unmounting the card you will need to
swapoff /mnt/swap/swapfile
Happy pocket hacking!

Related posts:
Gcal: the ultra-powerful command line GNU calendar
Unpacking my Ben NanoNote
My first port to the Ben NanoNote: gnugo
Another Ben Nanonote port: pmars
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!

23/11/2010 - 20 Best Posts I Have Read This October, 2010


Here comes my set of interesting blog posts for October 2010. As usual, they are divided in Programming (and related), Self-improvement (and related) and Blogging (and related). This month there are 7 posts in Programming, 10 in Self-improvement and 3 in Blogging. Enjoy!

Programming, Software and Related

Cleaning my favourite items from the RSS reader, I found a few from Matt Might's blog from when I started reading it. 3 shell scripts: Kill weasel words, avoid the passive, eliminate duplicates is a post where Matt turns himself into a shell script to advise his PhD students on writing "better" academic English. A good addition to your real life advisor! Another one, 7 Lines of Code, 3 Minutes: Implement a Programming Language. I have a long time longing for writing my own programming language... but still didn't take the plunge. And looks like it won't be this year, it has to be presented just before March (for language-related reasons).

30 Lessons Learned in 10 Years of Computing
is a collection of thoughts from making it work: musings of the math gladiator. I think that making your IDE really marked his life, he talks about this twice.

As usual, Re: Factor has a lot of good posts to share here: Presentations in Factor shows a way to make slideshows from within Factor. Syntax Highlighting, writing a syntax highlighter on top of the Factor REPL. And Text-to-PDF, how to convert text files into PDF files with Factor. Isn't Factor fantastic? I have to finish Thinking Forth! If I hadn't so many things to do...

My twitter friend John Metcalf has a blog named Retro Programming, where he uses to talk about the programming game Core War. And this months post is about just this: Core War - The King of Programming Games.

Self-improvement, Productivity, Life-Style and Similar

Did You Hear What I Said? From Ali Luke in Aliventures ponders about the difference between what we hear and what other people are really saying. Are you guilty of making things up to realise later that all was far simpler, just because you didn't care to listen carefully?

From the archives of Luciano Passuello's Litemind, How to Be a Human Calendar. I gave it a try, and although I didn't have a particular interest in being one, it was a good read, and the explanations in the post are crystal clear: give it a look if you want to be a human calendar.

I enjoyed quite a lot Tim Brownson's guest post in Charlie Gilkey's Productive Flourishing, How to Bounce Out of Bed Every Morning. I don't believe in neuro-linguistic programming, but the possibilities if it works can be amazing. Another post from this blog is It's All Been Done Before. So What? Don't think that because everything is done you can't innovate. There were a lot of MP3 players before the iPod... Innovation always wins.

Ryan Waggoner's blog makes his way into my list this month (and will repeat a lot more, November has already 4 more from him) with one post, Is There Such a Thing as a Sustainable To-Do System? Ryan is a Getting Things Done guy and I am not... But he makes good points in favour of it. Go read it if you are a GTDer or have been one.

My friend Fabian Kruse from The Friendly Anarchist is aiming at making a home run for good posts this year... He only needs one in December to make it (he already has two lined for November's post!). This months are Stay Hungry, Stay Foolish and The "Do Whatever You Want" Approach to Getting Things Done. Ways to improve, ways to motivate and the anarchistic way to be productive. Go read them now!

Tempered Steel, by Dragos Roua is a very good post. I think he touched upon something I was already thinking (and that came around as my own post What It Takes to Be a Winner? Embracing Failure). Pressure, correctly managed will turn you into a stronger you. Don't let it break you down. From Dragos' blog I also recommend (wink wink) Why the 80/20 Rule Can Make You Less Productive: my take on "anarchistic productivity". Treat it like a rule, but break it often when trying to be productive.

Joel Runyon's The Blog of Impossible Things makes his debut (although I have been following it through Joel's twitter for a long while) with a post where I'm kind of featured... When Do You Give Up? (Part 2) [Or How To Stop Banging Your Head Against The Wall and Become a Champion]. It is a follow up to Part 1, where he shares his story with playing basketball and asks his readers for their view. I shared mine and was one of the 5 comments that made to part 2.

Blogging and Other Internet Stuff

Blogging Tips from Pro Triathletes is a guest post by Mike CJ in ProBlogger, comparing your blogging career with a triathlon: devil is in the details, take care of them to improve your blogging.

Earning Extra Income (from a Small Blog) is Get Rich Slowly's nice blogging tip. Nothing new here: selling books... the twist is on Amazon. Check it out!

My main motivation to print out The Art of War as an A6 booklet came from reading this post by Max Klein: How to Be Successful: Steal Ideas and Copy People. I don't share his view, although the post is good, and the idea worth thinking about at least. And re-reading the Art of War is a reason good enough.



Did you enjoy this list? Share it!

If you want to be ahead of my Best Posts, subscribe to my Delicious page, where I post them as I read them.

Related posts:
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!

31/10/2010 - 30 Best Posts I Have Read This September, 2010


Just in time I finish my "Best of" for September, November is already there for the list of October. This month comes with a little less posts than the previous: 20 for productivity, time management & lifestyle, 2 for programming and 8 for blogging and internet stuff.

Productivity, Time Management & Lifestyle

In September I discovered Luciano Passuello's productivity & time management blog, Litemind, and since then I've been digging into its archives. Wonderful content! I recommend (to name just a few) Overcome Fear of Failure (part I) (part II is also good), Time Budget: An Easy Way to Avoid Prioritization Dilemmas and Keep Your Life Balanced, 6 Productivity Principles to Live By (My Personal Productivity Manifesto), Will-Do Lists and Topics du Jour: Give Your Life Direction in Less than 10 Minutes a Day. I think you realise my point: the content in this blog is really good. And its design... I "stole" the capitalized letter at the start of the post for my blog!

From Freestyle Mind comes a post I liked a lot... I wrote it! It was my first guest post for Oscar. Have You Bent Your Spoon Yet? is how we titled it in the end, my first idea was Bend a Spoon Daily. Give it a read!

I've read a little of Zen Habits lately (not counting his book, Zen to Done, click here to view more details), not on time but from the archives. Who knows why... I think the most useful content in ZH is the oldest. For example, Haiku Productivity: Limit Your Work Week, 7 Powerful Steps to Overcoming Resistance and Actually Getting Stuff Done, The Simple, Ridiculously Useful Guide to Earning a Living from Your Passion and Top 30 Tips for Staying Productive and Sane While Working From Home.

The Novel That Is Going Nowhere is a small piece from Life As Fiction written by my tweep Nicholas Ochiel. Worth a read. Go and do it!

The Berlin Street Artist: A Parable of Passion and Practice from Uncornered Market is an enlightening interview to a street performer, focusing on how to be the best. Practice, practice, practice! I am not a regular follower of this blog, Fabian from The Friendly Anarchist posted it to twitter and I found it too good to miss out.

The Friendly Anarchist also makes September's list with The (Un)Productive Night Owl: Fabian's experiments in being a... late riser. In this world of productive early risers wannabes, Fabian tries to just work at night. Succeeds? Read this post and the follow-ups!

Marc and Angel Hack Life offer an interesting method to improve your memory in A Bedtime Memory Building Exercise. Every time, before going to sleep just give a few minutes to remember everything you did today. You will get better with time, and your memory will be a lot better, you'll see.

Charlie Gilkey's Productive Flourishing has an interesting post this month: What Are You Going to Do Less Of? If you want to do more, try to do less. Our schedules are already overwhelmingly full. Simplify!

Study Hacks writes this month about The Romantic Scholar: A New Approach to Student Life. A way of living students life like in the Newton, Herschel or Leibniz era, when everyone enjoyed their learning in a very wide array of subjects. The Danger of the Dream Job Delusionis an amazing overview of a widespread problem: the dream job. Job dissatisfaction is growing in 10%, just because some people get to work in things they love and start spreading that your job is boring and theirs is awesome.

I discovered a blog/news aggregator called The 99 Percent in September, just through a post I really enjoyed: How Analog Rituals Can Amp Your Productivity

Another blog I don't follow usually is Sid Savara's blog, but I get some of his good content through twitter anyway. In September I read the quite interesting Note Taking Strategies. Probably it is too late for me, as I don't take notes any more (in a presentation you don't "take notes", you just scribble some thoughts).

Another blog that I don't follow but I get good content anyway is The Skool Of Life, from Srinivasa Rao, and the post in question was The Late Bloomer Syndrome. In it, he explores the fact that you don't need to be an ace and do everything as soon as you can. And I like his image of surfing.

Programming, Computers and Similar

I read a very interesting piece in A Programmer's Place titled Who Killed Prolog? with a very interesting timeline on why Prolog died in the AI debacle in the end of the 80s and beginning of the 90s.

This month Re: Factor only offered me an interesting piece: Floating Point Fractions. Math tricks with Factor are always interesting to read.

Blogging and Internet Stuff

Two guest posts I wrote for The Copy Paste Blog: Optimising Your Blogging Workflow With Pen and Paper (or an iPhone) and Your Post Is Finished. Now? Blog Promotion Time. In them I try to set an effective writing strategy (in the first) and an effective promotion strategy (in the second). Of course, they are worth a read, at least from my own perspective.

This month comes with a lot of content from ProBlogger. Two articles on blogging burnout: Avoid Blogger Burnout: 5 Tips to Save Your Sanity and How to Re-Ignite Your Blogging Fire When You Are Feeling Burnt Out (this one a guest post from Glen Allsopp, from ViperChill). Two very interesting pieces worth a read: every blogger has been there at least once. Or will be. Another interesting piece (more so now that I am considering writing one) is 11 Ways to Convince Readers to Buy Your eBook. Still untested, but in a few years I will be able to tell you if they worked or not. And a piece I enjoyed a lot reading is 5 Ways to Blog Like Bruce Lee. Amazingly, it comes as a guest post from a blogger without a blog!

Another post from Freestyle Mind, this one from Oscar is really an interesting concept: The Twitter Salesman. Can you use twitter to sell affiliate products, solely? Well, you can write bots, but they will get banned. But works, mostly.

From Get Rich Slowly this month we have Growing Your Human Capital: 11 Ways to Boost Your Income-Producing Ability (guest post from Robert Brokamp from The Motley Fool). Just common sense, written down. Always useful as a reminder when we have a stable job.



Did you enjoy this list? Share it!

Related posts:

The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!

10/10/2010 - 33 Best Posts I Have Read This August, 2010


After my previous Best Posts I Have Read This June and July, I decided to keep collecting my best reads every two months. And now, October arrived waiting for my post on August and September... But as I reviewed my feed reader and Read It Later list, I was overwhelmed, and felt like you would be, too. So here comes the best posts I have read this August. They are not only posts published in August, but also posts I have discovered (or maybe re-read) this month according to my Read It Later. I hope you enjoy them. I have split them in 3 categories: Self-improvement and the likes (19 posts), programming, technology and related (10 posts) and blogging tips (4 posts). Almost all links in authors name point to their twitter profiles.


Self-improvement and the likes

Dragos Roua's blog (his twitter: Dragos Roua) has been gathering a lot of attention this August, by its wonderful series of posts titled How to Build Reputation With a Blog, in particular I enjoyed a lot the post Write Constantly in this series. It is a subject I can't agree more, as I already wrote a post about continuous improvement, which implies working every day in a particular big project. Also of note was his 3 Ebooks to Read This Summer, where he reviews 3 ebooks as the title promises. I must confess I have not read any of these, but the reviews look good (Dragos is a good reviewer), and as I also follow the blogs of 2 of these 3 writers, I can also say they should be good. This month I also took the time to delve into its archives, finding the very worthwhile Put Your Blog Into a Mind Map. It is good advice, but I am not sure I can make it: my blog has now more than 250 (English) entries... But I think it is worthy of consideration, as I have the feeling I am not the only one forgetting what they have blogged about.

I'm a huge fan of Oscar del Ben's blog FreestyleMind, and August saw one post that I really digged. How to Write Early in the Morning was particularly to the point because at that time I was in the middle of an early raiser trial. I didn't succeed completely, but I was blog writing when I got up, thus it was just the thing.

Oddly enough, I am a reader of Get Rich Slowly. Living outside of the US with a completely different financial system means that I am not interested in most of its posts, but some are good anyway, and this month saw one of these. Action Not Words: The Difference Between Talkers and Doers was an enlightening read. I have been suffering talk-itis for a long time, and from reading the article I came up with what could me my new mantra, "Be a doer, not a talker".

This month Write To Done featured a very interesting article, Why You Should Shoot Adverbs on Sight. This is a very important advice, coming also from the fantastic book On Writing Well, a book about writing I just love.

Goal Setting Secret – How to Achieve Any Goal is an interesting read from Think Simple Now, long enough to even keep me from summarising it. A set of tips, most pretty good (one is do it daily) to help you effectively achieve any goal.

Do you follow The Friendly Anarchist by Fabian Kruse? You should. Fabian writes pretty long posts (for our current world standards, they are 50% longer than mine, I think) really well, usually with very compelling content. This August he posted Time Independence: A Personal Decision. The first line of this post goes like 'Much of the anxiety in our modern societies comes from the feeling that there is not enough time.' and it only gets better. Change your frame on time management, on time anxiety, on decision making. You have the choice.

Dumb Little Man (also known as DLM) has two interesting posts for this month. In How to Find Enough Time to Do Everything You Want, guest writer Ali Hale (from Aliventures) gives a list of steps to free your schedule and start doing what matters. Five Surefire Ways to Strengthen Your Willpower (guest post by Marelisa Fábrega, from Abundance Blog) is a good addition to the previous post: once you are free to do what you want, start really doing it. Because the most important the task, the more we procrastinate on it (read my review of War of Art by Steven Pressfield...).

I kept on reading Ali Hale's posts in DLM, and then decided to subscribe to his blog, Aliventures. And August saw the posting of When Overwhelm Hits Hard, a post trying to reduce your stress levels. If you are starting to feel overwhelmed, read it.

This titled caught me: How to complete your PhD (or any large project): Hard and soft deadlines, and the Martini Method. It came from Academic Productivity, something like a time management and productivity/self-improvement blog for academics. Very interesting (as it is similar to what I write here!).

I don't remember where I catched Discipline: Be the Machine, from a blog named Yield Thought. I guess it was either Reddit or Hacker News. But it was an interesting read on kicking your discipline, and it even has a picture of Paul Graham on it.

Productive Flourishing, Charlie Gilkey's blog has a great log of productivity advice. Coming out in August were Effectiveness Is All About Managing Your Time, Energy, and Attention and Are you at the helm or running around the ship?. The first collects some thoughts on effective time management, very important if you are not really tight focused on streamlining your schedule, and the second is a call to action to take reign of your life.

Jonathan Mead's Illuminated Mind also appeared in my RSS reader blog roll starting August and has one entry for this month (and more to come). In Are You All In?, Jonathan questions our use of language, from being partially to totally committed. Aim for the Moon, and hit it.

Matt Might has a wonderful blog with tips for PhD students and academics alike. This August he had 3 inspiring posts: 3 Qualities of Successful PhD Students (Psss: They are perseverance, tenacity and cogency), 6 Blog Tips for Busy Academics and The Illustrated Guide to a PhD. They are 3 really worthy reads if you are into research.

Programming, technology and related

As usual, Re: Factor has done some interesting stuff this month, if you enjoy stack based programming languages you should follow it. Among all his very interesting subjects, three posts caught my eye (and urge me to give up Forth and start Factor...). In Fat Arrows, some syntactic sugar (as the author defines it) is added to Factor to implement fat arrows, which indeed, are syntactic sugar. How they work? They are just a way to create 2-vectors without using the usual vector notation, thus a => b generates the 2-tuple {a b}. As usual with Forth related languages, adding syntactic sugar is painless and wonderful, more so in Factor, as this post shows. Calculator with GUI is completely descriptive: creating a simple calculator with a graphical user interface, in Factor. It is so easy I almost think it should be taught in schools! To end the month, Hello, Web! introduces the Factor web server, which incidentally powers all the Factor language homepage and related pages.

Another programming blog I follow, PhobosLab wrote a post about his entry in the JS1K contest (a contest for programs in JavaScript with less than 1K code), with a JavaScript syntax highlighter and quine (a program that outputs its own source code). Neat!

Citizen428.blog() is another programming blog with interesting and bizarre ideas (sometimes). This August I read there Re-wiring My Brain: APL and J and 30 Free Programming Ebooks. The first, a review of the APL programming language (J is a modern dialect of it), and the second is self descriptive. The selection is really good, covering a lot of different language paradigms.

Study Hacks is a blog... well, about Study Hacks. Getting the most out of your learning. And has a lot of interesting (and long!) content, this August specifically: Beyond the 10000 Hour Rule: Richard Hamming and The Messy Art of Becoming Great. In it the author discusses Richard Hamming's famous talk You and Your Research. He doesn't love it, I do.

I read a very interesting post (linked through Hacker News) in Dan's Data titled Next Stop: Clay Tablets about using QR-codes as data transmission systems. I think I'll print out the TeX sources of my thesis like this when I finish. It will look weird.

I am not sure how I found out Optimizing Your Wallet at Coding Horror (as it is a post from April), but it was a bizarre read. More so as my wallet looks suspiciously similar to Jeff's previous wallet.

Fire and Motion, from Joel On Software was enlightening. In it, Joel explains his workflow: there are times when he just can't do anything. And then one day, you start working, enter the flow and get a full load of s**t done. I've felt this way so many times...

Blogging tips

ProBlogger from Darren Rowse also has a share of the pie, as a blog publishing daily has more chances. The post, Guess What? Bloggers – You and Me – Are Publishers, and We Should Act Like It. (So Let’s Not Be Jerks when Someone Sends Us a Guest Post!) has still some touch to me, as in August 4th I submitted a guest post to a big blog (let's not say its full name... its acronym goes like DLM) and I am still waiting for some kind of answer. The post How to Make Your Blog Addictive Like World of Warcraft was very fun to read, and it gave pretty interesting insides in catching new readers. Finally, a good rant read has been Why Link Exchanges Are Like Mosquitoes. Go and read it, it is just a mix of rant and personal thoughts, but it is definitely interesting.

ViperChill is Glen Allsopp's blogging blog, and although I am not very fond of blogging blogs,The Most Effective Way to Increase RSS Subscribers was very good, I have to say.

Related posts:

The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!

06/10/2010 - And e/2 Appears from Nowhere! (Follow up to 'And e Appears from Nowhere')


You may remember a post I wrote a month ago titled And e Appears from Nowhere. It was based (through some blogs I read) on a footnote from Prime Obsession (Amazon affiliate link) by John Derbyshire. The footnote reads:
Here is an example of e turning up unexpectedly. Select a random number between 0 and 1. Now select another and add it to the first. Keep doing this, piling on random numbers. How many random numbers, on average, do you need to make the total greater than 1? The answer is e. (See sketch of proof in the previous post)
I wrote some clojure code to test it, and even found the proof lying around. It was very interesting.., and the post was quite visited. A few weeks ago I got a comment in that post, hinting at another bizarre happening.

If you follow the previous procedure, in each step you will get a number greater than 1, say x(i) (for the step i). What is the average of x(i)? This is the average number you get greater than one in each step. Well, James (he made the comment) had a bug in his first try and computed that. He found out that this average looks suspiciously close to e/2.

I also checked, and found the same. The modifications from my previous code were minimal:
(defn PlusRandom2 [initial steps]
; Initial and step start at 0. Will return how many times it takes
; to be greater than 1
(if (> initial 1)
initial
(PlusRandom2 (+ initial (rand)) (+ 1 steps))))

(defn AveragePlusRandom2 [iterates accumulator]
; Takes the average of "iterates" iterations of the previous
; function. Accumulator should start at 0
(loop [n iterates accumulator 0]
(if (= n 0)
(/ (float accumulator) iterates)
(recur (dec n) (+ accumulator (PlusRandom2 0 0))))))

(def ehalf 1.359140914229522617680143736)
And one run yields
(AveragePlusRandom2 10000 0)
1.3563358
And indeed, a few runs comparing to e/2 return
(- (AveragePlusRandom2 10000000 0) ehalf)
-1.0387256539901024E-4
(- (AveragePlusRandom2 100000000 0) ehalf)
-4.57122720320946E-6
The problem now is, how to prove it? I have been thinking it over for a while, and I guess it has something to do with the central limit theorem (maybe indirectly via sums of uniform distributions). But my knowledge of probability is far below the requirements, or maybe isn't, but the time I can allot to keep trying is scarce. If any reader takes the plunge and finds a proof, I'll be very happy to share it here as a guest post. Prepare your pens!

Related posts:
9 programming books I have read and somewhat liked...
C code juicer: detecting copied programming assignments
Cron, diff & wget: Watch changes in a webpage
8 reasons for re-inventing the wheel as a programmer
Approximating images with randomly placed translucent triangles
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!

23/08/2010 - Power nap via the command line in Linux/Mac OS
A month ago, Oscar del Ben posted an interesting tip in his blog, How to Get a Quick 5-10 Minutes Nap Without Using an Alarm Clock. His idea is simple: pick a spoon on your hand. Once you fall asleep, your grip will relax, the spoon falls and you awaken with the sound. I found it amusing and interesting, as I am one of those types who feel really well after a 10 minutes sleep.

The catch? You sleep almost nothing, with this method. You doze, and when you enter real sleep mode, you awaken. It can refresh your mind, but it does not qualify as a Power nap:

Scientific experiments (see Benefits section below) and anecdotal evidence suggest that an average power nap duration of around 15-30 minutes is most effective. Any more time, and the body enters into its usual sleep cycle. People who regularly take power naps may develop a good idea of what duration works best for them, as well as what tools, environment, position, and associated factors help induce the best results. Others may prefer to take power naps regularly even if their schedules allow a full night's sleep.

Thus the ideal sleep length is around 15 minutes. How do you get yourself to sleep 15 minutes? I find it very hard, because a power nap is ideally taken after lunch, when you may be sleepy, but not that sleepy to fall asleep instantly, less if you are sitting in your office chair. You can estimate how long it takes (for you!) to fall asleep, but from my own experience, this varies greatly. I usually take around 5-15 minutes at night, but can take around 20 minutes when it is daytime. How to sleep for exactly 15 minutes after falling asleep?

The solution is pretty easy: what do you have in your office, always on? Yes, your computer. If you are a Windows user, I am sorry: my solution works in Linux or Windows. You can use Cygwin, or fight your way to implement the same in your own platform. I encourage you to do so!

What is this solution? A bash script, with several command line parameters. Once started, it waits for a second for a keypress. If a key is pressed constantly, i.e. you have your finger over a key, nothing happens. As soon as you release the key, a counter starts running, and when it finishes your computer plays a beeping sound or a file of your choice.

The idea is that you get in a comfortable position in your desktop, start the script and keep your finger pressing on a key. You have to be in a position such that when you fall asleep, your finger falls of the keybord. One option is to slouch in your chair with your arm extended pressing the space bar, when you fall asleep, your hand will fall over your desktop. Another option is crossing your arms over your desktop, with your head over them and the keyboard on a side. You just press enter with your hand and when you fall asleep your hand will fall.

When you sleep, the counter keeps ticking, and when it ends, music (or console beeps) awaken you.

Usage is ./nap.sh naptime "musicplayer" or ./nap.sh test "musicplayer". If musicplayer is empty, it will use the system bell, and test is used to be sure it will do something after the selected time. As of this writing, the music file is encoded in the script (feel free to change it to some file you like!), and you have to use the music player best suited to your system. In my MacBook, I would use open as music player, in my NanoNote or netbook, madplay (command line music player) with the option -a -30 to adjust the volume.

An example use (the way I use it in my NanoNote) is ./nap.sh 15 "madplay -a -20"

You can get the most up-to-date script from my Google Code repository here: nap.sh. In a future version, I will add a safety control to beep after naptime+safetymargin minutes, to solve the problem of pressing a key and then falling asleep over the keyboard.

If you have any suggestion or improvement to the script, or have used it please drop by and leave a comment! Also, share with your friends in Reddit, HackerNews, Digg or StumbleUpon.

Related posts:
9 programming books I have read and somewhat liked...
Linux is a time killer (follow-up – Ditching Ubuntu: Arch Linux in my Acer Aspire One)
My first port to the Ben NanoNote: gnugo
Power to the command line
Two weeks, still loving Fluxbox
Three dee (3-dimensional file system browsers review)
Gcal: the ultra-powerful command line GNU calendar
Acer Aspire One 8.9' + Ubuntu + Fluxbox
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



17/08/2010 - Which programming language should I learn next?
I have a problem: I know quite a lot different programming languages, but I am proficient only in two. I'd love to choose a few to concentrate and build my skills up, but I don't know which. Read on for background and possible candidates.

Since I discovered there were different programming languages, I've liked learning different ones. It is the same with languages, I love learning new languages.

But I have focus problems, and after a while I just give up and forget half of what I learnt. I can write in C and Lisp without much problems, except for looking some function up in the Lisp Hyperspec and a recurring forgetting of how do is used. But my working knowledge quickly decreases:

Awk: wrote several scripts to solve some problems, but could not write anything out of the cold. Useful enough for what I do in awk.

Bash: wrote more than a few scripts. As bash is not hat deep I guess I know enough for my uses.

Forth: wrote a little to see how it goes, but not enough. I like stack based languages.

Java: I have a little working knowledge and a small cheat-sheet I wrote and used. I had a course on databases where we had to write a mail back-end (SQL) and write accessing user code in Java. Learnt enough to do it, no more as I had no time back then.

JavaScript: wrote several naïve scripts. They work, but are pretty amateurish.

PostScript: wrote a few programs and a small tutorial. Used too little to work from scratch, but enough for what I actually use it.

Python: wrote a few small programs and read a few times a book. Didn't code for long enough to write programs out from scratch.

I want to pick one language from this list or new (or a few, no more than 3) and really get into them, up to being able to write code out of the blue. If possible, not too amateurish code. In my free coding I usually do fun things... I have half a raytracer and half a SIRDS generator, both in Lisp. I also have an image approximator by triangles, stochastic, in C. These are slow programs, requiring quite a work of optimising. Now, I don't know which languages I should choose next, less in view of what I use them for. General usefulness is welcome (if I can get something out of a language I learn in my free time, the better). Here are the options I am considering, together with pro's and cons. If you think something is missing for your favourite language, please tell me so!

Disclaimer:
I consider being hyped as a pro feature, because it means it has a wide array of developers and documentation, not because I want to learn the top of the wave language right now.

Clojure

Pros (Clojure): It is a Lisp based language, and this means that I have a great deal already learned. It is also being hyped and talked about. I have already written some kind of hello world program in it, in my post And 'e' appears from nowhere I approximated e using Clojure. Google Trends indicate the language is currently in a local decline from an increasing trend.

Cons (Clojure): But I know very little Java, thus I could not get the full power of the language straight out. Also, I don't know what I would use it for. How fast is Clojure? How does its memory management work?

Factor

Pros (Factor): Factor is a Forth like language with a deep set of libraries. It is currently being actively developed.
Cons (Factor): I feel like I don't know enough of he 'mother' language, Forth, to switch to a derivative. Not enough volume to show in Google Trends.

Go

Pros (Go): Written by Google and actively developed. Much hyped. It is very much C-like.
Cons (Go): Google Wave was killed, it could also happen to Go. Also, I guess pure C would be faster for a C-like compiled language, thus I could use C instead. Google Trends doesn't seem to indicate it is very interesting. A big spike on anouncement and a bumpy constant curve afterwards.

Ruby

Pros (Ruby): Hyped a lot. Looks like everyone around is using it for something. Steady constant curve in Google Trends.

Cons (Ruby): As of now, I don't even know if it is compiled or interpreted, or what it is good for.

JavaScript (in depth with jQuery and node.js)

Pros (JavaScript): I could write a few iPhone apps I have in mind in JS. I know a little JS already, and it is pretty much an interesting language.
Cons (JavaScript): Too much work for too little output, maybe. I don't even know if there are good references for in-depth Javascript, or for jQuery/node.js. Declining (heavily) in Google Trends, but jQuery and node.js are rising strongly.

Python

Pros (Python): From what I already read and programmed, it feels like a wonderful language, very thought out. His creator even has a beard!
Cons (Python): I don't know what uses I could put it for. Speed would be an issue for a lot of programs I may write, but not for all of them, of course. Slight decline in Google Trends.

Thus, what do you suggest? If I was to choose right now, I would probably pick Python, Forth and Clojure (although I'm afraid of how much Java I need for it). Which one would you pick? Or which 2-3 languages do you suggest for me? Do you think 3 languages at the same (probably not same as in the same day or week) time are too many?



The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



13/08/2010 - Two images, an apology and a 'new' blog





Don't worry, I won't start posting just images, I have a reason (but not a good reason) for posting just two images and a small blurb of text today.

For posting just images, I have my new Random Thoughts side blog, where I post shorter than a full post but longer than a tweet ideas, along with some images I also run here. It is powered by Tumblr, which I see as a pretty good blogging platform, more so for images or short posts like I've done. You can subscribe to Random Thoughts blog here.

I will be out for this weekend... and didn't finish enough blog posts to cover for it. Ouch! I have a big idea drawer overflowing, but these days I was hacking away a program to test a mathematical idea I have... Here you can see two of the 99 images it generated.

The idea I had looks as if it is true (i.e. the images look remarkably similar in the main central shape, don't you think?), now I need to find a proof for it.

If you happen to like these two pictures, I am considering running again the program with a slightly smaller zoom, to be able to create desktop wallpapers with the spiralling thing in the middle. I have done it previously with my Siegel disk images and my Douady rabbit images.

Drop a comment if you are interested in these wallpapers, if someone shows up I'll do them and post soon (after the weekend out+calculation time, roughly, Tuesday). And then subscribe to my RSS to keep in touch!

Related posts:
9 programming books I have read and somewhat liked...
And "e" appears from nowhere!
Power to the command line
Two weeks, still loving Fluxbox
Three dee (3-dimensional file system browsers review)
Gcal: the ultra-powerful command line GNU calendar
Acer Aspire One 8.9' + Ubuntu + Fluxbox
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



05/08/2010 - And e appears from nowhere!
As I posted in Best Posts I Have Read in June and July, I liked a lot a numerical experiment in Re: Factor, a blog about learning the Factor programming language. The idea comes from another blog, this time about Clojure (a Lisp dialect running in the Java Virtual Machine), based on a footnote from Prime Obsession (Amazon affiliate link) by John Derbyshire. The footnote reads:
Here is an example of e turning up unexpectedly. Select a random number between 0 and 1. Now select another and add it to the first. Keep doing this, piling on random numbers. How many random numbers, on average, do you need to make the total greater than 1? The answer is e. (See sketch of proof below)
In the original post, a quick Clojure implementation is programmed, and in Re: Factor, a Factor implementation is added. When I originally read the post, I missed that the original version was in Clojure, and decided I was good to go by writing a Common Lisp version. I'd prefer a Forth version, but as Factor is very close to Forth, it was close to cheating. Keep in mind that I wasn't aware that the first version was in Clojure!

I switched to my emacs session and started slime. To my surprise, slime fired up Clojure, instead of Common Lisp! I didn't remember that a few months back I got interested in this language, and set up slime to start Clojure. Oh well. My first program in Clojure, I thought.

The first function was easy to define. A function which picks random numbers from 0 to 1 and keeps on adding them until they are bigger than 1. As this was my first Clojure project, I didn't want to bother learning how Clojure loops work... I wrote a recursive version. I did not try to make it even tail recursive:
(defn PlusRandom [initial steps]
; Initial and step start at 0. Will return how many times it takes
; to be greater than 1
(if (> initial 1)
steps
(PlusRandom (+ initial (rand)) (+ 1 steps))))
Ugly, I can hear you mutter. Maybe, but this was my first ever Clojure function, and it is almost the same I would write in Lisp. Moreover, after inspection, it looks like it is indeed tail recursive. It still amazes me that it worked! Of course, this can be improved, at least to make it more clear, with an auxiliary function:

(defn PlusRandomAux [initial steps]
; Initial and step start at 0.
; Will return how many times
; it takes to be greater than
; 1
(if (> initial 1)
steps
(recur (PlusRandomAux (+ initial (rand)) (+ 1 steps)))))

(defn PlusRandom []
(PlusRandomAux))
After this function, we have to take averages of what the last function returns, for a large number of iterates. My first version of this function was recursive, too. But it blew away the stack, which is not good. I conceded, and looked at Clojure's loop construct, and the result was
(defn AveragePlusRandom [iterates]
; Takes the average of "iterates" iterations of the previous
; function. Accumulator should start at 0
(loop [n iterates accumulator 0]
(if (= n 0)
(double (/ accumulator iterates))
(recur (dec n) (+ accumulator (PlusRandom))))))
Now, I look at recur, because it looks odd. Turns out, Clojure does no tail call optimization! Big fail for my PlusRandom function. Rewritten with a loop construct, with recur:
(defn PlusRandom []
(loop [cnt 0 accumulator 0]
(if (> accumulator 1) cnt
(recur (+ 1 cnt) (+ accumulator (rand))))))
for a gain of an 11% execution time (from 3.9 to 3.5 seconds). It looks like too little for a tail call... but as most calls end either with 2 or 3 recursions, the improvement should be small? No! It turns out that the auxiliary function call adds an overhead... which accounts for a change from 3.9 seconds to 3.7 seconds! And the real improvement from tail call optimization is just a mere 6%.

Of course, if you want a quick program, you do it in C, if you want to program quick, you do in (insert your favourite quick language). The convergence of this scheme is too slow, too. But the answer comes from the mathematical proof.
user=> (time (AveragePlusRandom 100000))
"Elapsed time: 421.315711 msecs"
2.71938
user=> (time (AveragePlusRandom 1000000))
"Elapsed time: 3445.167485 msecs"
2.718408
user=> (time (AveragePlusRandom 10000000))
"Elapsed time: 34082.850845 msecs"
2.7182318
By the way, the proof from the convergence to e is quite straightforward, but this morning I was in no mood for this kind of problem (I am not fond of counting or probabilities) and googled it. I found it in the Google cache for Derbyshire's book FAQ. I have edited a little his version

The first step is key. What's the probability that the sum of just two random numbers exceeds 1? The same probability as that a point (x,y) in the unit square (corners are at (0,0), (0,1), (1,0), and (1,1)), satisfied the inequality x + y >= 1. This is of course 1/2; draw the diagonal from (0,1) to (1,0), and it's the probability that (x,y) is in the upper right triangle.If you now go to three random numbers, the chance that their sum exceeds 1 is got by "cutting a slice" across the unit cube through the corners (1,0,0), (0,1,0), and (0,0,1). Observe that this pyramid (shown below) also cuts the unit square along the diagonal! This pyramid has volume 1/6, and the probability x + y + z >= 1 is then 1-1/6=5/6.


A little calculus, plus the principle of induction, shows that the pyramid (standard simplex) sliced off an n-dimensional unit cube by a hyper-plane through its corners (1,0,0,0,...), (0,1,0,0,...), (0,0,1,0,...),... has volume 1/n! If you don't believe it, you can check Wikipedia for the volume of the standard simplex.

The probability that a total of 1 is exceeded after adding n random numbers is therefore 1 - 1/n!The probability that a total of 1 is exceeded after adding n random numbers but not before is therefore (1 - 1/n!) - (1 - 1/(n - 1)!), which simplifies to (n - 1) / n! The expected number of selections until a total of 1 is exceeded is therefore the following sum:


The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



18/07/2010 - Featured in a podcast
While I was in Iceland, I started to see visits coming to my blog from www.hanselminutes.com... A site I didn't know about. As soon as I got a decent wifi coverage I went to see what it was about... And it was a podcast!

From his own description,
Hanselminutes is a weekly audio talk show with noted web developer and technologist Scott Hanselman and hosted by Carl Franklin. Scott discusses utilities and tools, gives practical how-to advice, and discusses ASP.NET or Windows issues and workarounds.
The interview where my blog was mentioned was Hanselminutiae Eight: Interview with Dan Fernandez, and the post they talked about was 8 reasons for reinventing the wheel as a programmer, after it appeared in HackerNews for a second time. I thought you could not re-post things to HackerNews, but it looks like I was wrong!

Related posts:
7 Links For Bloggers (à la ProBlogger)
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



06/06/2010 - On a productivity down
A week without writing here. A week with little thesis related work done. But it has also been a week with ideas and things and such. You know, two weeks ago I was in Dresden for a conference. Lots of parallel sessions, and quite a few time to think. This post is mostly a digest from my life bookmarks for these two weeks.

Several complex dynamic ideas: Unrelated to my thesis, but I've been thinking about them these days. I even rewrote a program to test an idea, and did some calculations yesterday. Looks like it will work out, and I will be able to finish some things.

Heard a few interesting plenary talks: Cloaking, invisibility and inverse problems by Gunther Uhlman, who talked about theoretical solutions for creating invisible objects, as well as some real life applications like a cloaking system against tsunamis (by making an island invisible to water waves). Recovery of Data Matrices from Incomplete and Corrupted Entries by Emmanuel Candes was really interesting. He talks about a method (and related theorems) for recovering a full matrix from a very small sample of it (one of his examples was a 30000x30000 low rank matrix, where he only sampled 0.4% of its entries, and recovered the full matrix with a very low error. Really interesting! You can read the preprint in ArXiV.

Heard also a few interesting short talks: partial differential equations related talks, mostly, also one about neuron networks. Had an idea about the wake-sleep cycle problem as a synchronization problem.

Ate interesting things: I didn't remember spätzle were that good. Should cook some day. Also tasted Pho bo soup in a Vietnamese place, and it is really a wonderful soup.



After coming back, I have also done a few things, but not so much related to my thesis

Installed Debian on the Ben NanoNote: Now I use emacs and org mode to keep my life bookmarking. In the nanonote you need emacs23-nox, emacs22-nox complains about little memory, don't really know why (it only uses 7mb, there are still a lot left!). Also, if you happen to find a bus error while reading the aptitude package list (which will appear as Bus errorackage list 0% in your console), first try to delete /var/cache/apt/*.bin (or /var/apt/cache/*.bin) and apt-get update. If it does not work... I advise you to reinstall your debian system. I was not able to solve it, and ditched Lenny for Sid, then it worked well. But if you Ctrl-C while apt-getting something, you'll find the bus error again, which can be solved by removing all these bin files.

Ordered, got and tried a pair of Fivefinger KSO's: These are a barefoot-like running shoes. You can buy your pair from Amazon (affiliate link). They are really good, running never felt like this before.

Had some origami ideas: It was really long since I created something. Have a few ideas, or at least, a few things worth sharing.

Decided to finish unfinished things: I have one lemma left since my DEA (advanced courses) days. I need to finish its proof. I am also considering having another look at my previous thesis topic. I don't feel like keeping things undone is good for my mental health.

Discovered an interesting site: CreateSpace.com for self-publishing and selling in Amazon.

Baked an interesting dish: Very close to Aeblekage, a Danish recipe I like a lot. Ate it today for breakfast.



I don't know when I will be able to post this coming week... I used to write my posts in advance (usually one Friday and two Saturday), but this week I have none! But don't think I am not here, this is just being out of sync.
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



20/05/2010 - The 100 most common words in Icelandic (more or less)
As you may already know, I'm travelling to Iceland this July, and started learning Icelandic a few months ago. It advances slowly but firmly, but I found a problem: when you are self-learning a new language, an invaluable tool is a list of most common words. I was able to find the 100 most common words, from a research paper (Íslenskur Orðasjóður - Building a large icelandic corpus). I don't want to dismiss their results, but for a published paper you can't count twice Hann and hann, or count f. as a word, I think. However, they explain the procedure in the paper, and it looks pretty good. Just that the list they give leaves a little to be desired, and I could not find a way to use the corpus they generated to get the frequency list.

I decided to get something different. First I thought of sampling a lot of Icelandic data (online news and such), but I didn't want to waste that much time... I downloaded is.wikipedia.org. A meagre 42Mb of compressed data! Well, it could be even smaller! The article sampled 142Gb. Truly amazing!

After I had the data, I wrote a small script that moved all html files to the same directory:
#/bin/bash

# Navigate through directory tree and copy all html files here

for FILE in $(find ./ -name *.html -type f); do
mv $FILE ./
done
My idea was then to cat all these files into one big html file, and then do word frequency analysis there. Problem: cat *.html > file does not work: *.html yielded too many results (around 60 thousand, I think). Instead of writing a script (the solution I should have used) I just cat-ed every letter as in cat A*.html > is-a.dat. I should have used a script similar to the one I created for the Christmas postcard:
for i in `seq 1 $FILES`;
do
let NUM=$i*$COLUMNS
ls *.jpg | head -n $NUM | tail -n $COLUMNS > F$i
done
This is the original code. In the file F$i I would have a list of all files I need to cat together. Anyway, I did it by hand. On my way to the end file, I found several letter combinations (Fokkur which means Categories, for instance) with a lot of pages, which cat also could not manage. (I think the problem was bash, more than cat) I removed them, because the Categories page could have a strong bias towards certain words.

Once I had this big WPislenska.dat file, it was just standard command line tricks (which I got from the Linux Cookbook) (where /lt is <>)
tr ' ' [return]
'' /lt WPislenska.dat | sort | uniq -c | sort -g -r /gt IslenskaWF-FromWP.dat
This turns spaces into returns, sorts alphabetically, counts unique words and orders in decreasing frequency order.

Now IslenskaWF-FromWP.dat contains word-frequency counts for data from this Wikipedia dump. The next step was the maddening one: removing all html entities, wikipedia words (like page, visitors, users...) and find the English translation, via Wiktionary, my Icelandic dictionary, my Icelandic learning course and Ubiquity.

The final result is this file, with the 100 most common words in Icelandic's Wikipedia. If you can help with some messy translation, feel free to add a comment in the comment box below.

Related posts:
9 programming books I have read and somewhat liked...
C code juicer: detecting copied programming assignments
Cron, diff & wget: Watch changes in a webpage
8 reasons for re-inventing the wheel as a programmer
Approximating images with randomly placed translucent triangles
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



12/05/2010 - Giving up the Forth raytracer
Today, while I was thinking of the best implementation solution for the vector operations, I realised that I am just not motivated by writing a raytracer in Forth. I'll have to find something more interesting, or at least, more Forth minded to work on.

If I want to raytrace, better to improve the Lisp raytracer, which is sitting idly in my Code/Lisp folder. Steps that will follow in the raytracer path:
  1. Code commenting: I almost forgot how the main loop was rolled
  2. Performance: ditch temporary image matrices and output straight RGB data. This means no antialiasing, out of the box without some tweaking. But I can tweak that with Monte Carlo methods
  3. New objects: Where are my triangles? I only have spheres. I need triangles, and triangle meshes.
  4. Triangle mesh importer: from a nice ASCII format generated by some open source program
  5. Transparency: My previous version had it, this new one, does not.
With respect to Forth, I asked the question on Reddit#Forth, and from the answers, I'll probably switch the Logical maze generator/solver into a Forth program (looking at my code, maybe it is better suited to it, even), with an incorporated player. Then, if it is nice and playable, I can write a small JavaScript Forth interpreter (or even easier, use just some kind of intermediate layer to translate Forth into JavaScript) to be able to play it on a webpage.

If you have any other suggestion for a Forth project, leave a comment here or in Reddit. Thanks!

Related posts:
9 programming books I have read and somewhat liked...
C code juicer: detecting copied programming assignments
Cron, diff & wget: Watch changes in a webpage
8 reasons for re-inventing the wheel as a programmer
Approximating images with randomly placed translucent triangles
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



01/05/2010 - My take on 'Logical mazes': Part 1
A few days ago I found Theseus and the Minotaur (Theseus, Theseus Lite non-affiliate iTunes links), an interesting puzzle for the iPod/iPhone. It is a member of the family of logic mazes. Logic mazes were invented by Robert Abbott, and the history of this interesting puzzle can be read here: 'Beyond Tetris' - Theseus and the Minotaur / Mummy Maze.



In this logical maze, you are Theseus, and can move one step everywhere not blocked by a wall in every turn. You have to reach the stairs before the Minotaur kills you. The Minotaur moves twice in every turn if it can. It will always try to move closer to you, and start by moving sideways. If it can't move, it will forfeit its turn.

With such simple rules, you can get mazes like the one above, and harder still. I wanted to do something similar, and had one or two ideas I'd like to implement. For this, I have written a random "maze" creator. For now, it just populates a NxM matrix with integer values, representing walls, and outputs an ASCII or Postscript version of the "maze". I have quoted maze because it still does not qualify as a maze. Below you can see the output. There's still a lot of work to do for it to be useful, of course.



It still has not any kind of test for closed loops or playability, it does not even place the player, enemy or end on it.

I had an internal debate on which language to implement this idea (and the following parts of this, which are the real maze generator and solution testers). I was thinking of either C or LISP, and settled for C. I am more fluent programming in C, moreover I had the doubt of what the speed constraints would be... when in doubt I go on and write it in C.

Related posts:
C code juicer: detecting copied programming assignments
Cron, diff & wget: Watch changes in a webpage
9 programming books I have read and somewhat liked...
8 reasons for re-inventing the wheel as a programmer
Approximating images with randomly placed translucent triangles

The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



17/04/2010 - 9 programming books I have read and somewhat liked
I am no professional programmer, and as such, have read just a few books on programming. This is no exhaustive list, as a few I have read and promptly forgotten, they are not up to be remembered. Others I am reading currently (Code Complete, Thinking Forth), and maybe they could make it into this list someday.



Kernighan & Ritchie's The C Programming Language

This was one of the first programming books I read on its fullest, and probably the only one I was really forced to do so. In my first year in the University, I was required to program in C, and previously I had only programmed in Basic (Visual Basic, GW-Basic, QBasic), Pascal (Turbo Pascal for MS-Dos and Windows). The first semester was ok, as we were taught the basics of the language, but I wanted more and started reading this book (loaned from the library). I liked it a lot, learn a lot and still think that its clear style and simple examples make it the best language specific programming book I will ever read... If we don't pit it against the following.



Paul Graham's ANSI Common Lisp

I learnt about Lisp a long, long time ago, but stalled on it until I stumbled upon Graham's site, and read his somewhat inspiring history behind Viaweb. Decided to give it a try, and have loved this language since then. My first buy from Amazon was this great book, and won't ever regret having bought it, would probably even be one of my 10 books to take to a desert island. Worth a buy, and even a second buy to give it as a present.





Friedman & Felleisen's The Little Lisper

A little and short gem among programming books, teaches recursion, some programming and a little subset of the Lisp language to anybody in a little number of pages. It is worth a read, even if you have been into programming for a long time.



Hunt & Thomas' The Pragmatic Programmer: From Journeyman to Master

This is probably one of the most recommended books on programming, and for a reason. Buy it. Now. And read it as soon as you have it.




Adobe's Chapter 8 of The Postscript Programming Language Reference Manual

The PPLRM is a huge book, with lots and lots of information. And yet, the only part I have ever used is Chapter 8: Operators. Almost anything can be found in here, if you know how stack-based languages go. Concise and to the point: is just a chapter on how operators work.

Buy on Amazon link (full book, of course)



Watt & Mangada's Basic para niños

This translates as Basic for kids, and it is a really lovely book. My first programming book, I learn a lot from it (although maybe too many goto's ;) It is a wonderful book for teaching programming to children between 5 and 11 years old. Do you readers know if there is something similar but up-to-date?

Not available to buy (AFAIK)


SAMS' Teach Yourself Python in 24 hours

Yes, I fell for the Teach Yourself X in N hours/days, but this book is pretty good. It feels more like an introduction to the language than one of those standard rubbish books. But after a while, you grow to hate the Mayan calendar examples the author loves, sadly.



Friedman & Felleisen's A Little Java: A Few Patterns

I though I should give a try to another Friedman-Felleisen combination. This is not as good as The Little Lisper, although it is not really bad. Just not the same type of book. Worth a read, but loaning from the library. I would not probably buy it.



Abelson & Sussman's The Structure and Interpretation of Computer Programs

A classic one... Known as The wizard book, for its cover. But I think I just read half of the book, from re-reading the Contents. Maybe I forgot everything? Focused on programming in Scheme, it teaches a lot about general computer science. Worth a read, but it is quite a hard read. It is freely available.

Buy on Amazon link

Free html version link



You may be also interested in
The "Related posts" method I use involves Javascript, thus it doesn't work in the RSS feed. To view related posts, please refer to the original article. Thanks!



11/04/2010 - Another Ben NanoNote port: pMARS


Ben Nanonote
Screenshot compositing, made with free software

Since I bought the Nanonote, I have been finding uses for it. Music player, note taker, voice recorder. I can also use it to start learning Python again, or Perl, which are (together with Lua) the languages currently installed by default.

After my first successful port (gnugo), I decided to try something else, and while idling at the train I thought that pMARS, the portable Memory Array Redcode Simulator was probably a good bet. If you don't know what I am talking about, you can check here. I had just started writing some test warriors, to see if I understood the code, but had too little time to test it working.

And as I thought, pMARS was not as hard as other programs have turned to., but without graphical output, just text based. Enough. What changes are needed?

Follow the steps in my previous tutorial, the changes needed fit in there.

The corresponding openwrt Makefile for pMARS should be
# Copyright (C) 2008 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
#

include $(TOPDIR)/rules.mk
## Package name
PKG_NAME:=pmars
## Package version we need (to match our download source)
PKG_VERSION:=0.9.2
## Release version (don't really needed, just for completeness)
PKG_RELEASE:=3
## Name of the file we will download, with the previous package name definitions
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
## Url with the PKG_SOURCE file
PKG_SOURCE_URL:=http://downloads.sourceforge.net/project/corewar/pMARS/0.9.2/pmars-0.9.2.tar.gz?use_mirror=mesh
## Program to uncompress the sources
PKG_CAT:=zcat

include $(INCLUDE_DIR)/package.mk
# Definition of the package, for adding to menuconfig and creating the ipkg.
define Package/pmars
TITLE:=Pmars
SECTION:=games
CATEGORY:=Games
URL:=
endef
# Directory where the sources will be uncompiled and built BUILD_DIR
# is an env variable from the OpenWrt toolchain, you don't need to set
# it.
PKG_BUILD_DIR:=$(BUILD_DIR)/pmars-$(PKG_VERSION)
# Long description of the package
define Package/pmars/description
PMARS
endef
# If ./configure does not need any options, leave it as follows

define Build/Prepare
$(call Build/Prepare/Default)
$(CP) $(PKG_BUILD_DIR)/src/* $(PKG_BUILD_DIR)/
endef

define Package/pmars/Build/Configure
$(call Build/Configure/Default)
endef

define Package/pmars/Build/Compile
$(call Build/Compile/Default)
endef

# Where will be copied the binaries and libraries after installation.
define Package/pmars/install
$(INSTALL_DIR) $(1)/usr/sbin
$(INSTALL_BIN) $(PKG_BUILD_DIR)/pmars $(1)/usr/sbin/
endef
# Final step for bulding the package
$(eval $(call BuildPackage,pmars))
And for it to work, you need to make some static patches to the Makefile in the src directory of the pmars package. Change the compiler for
CC = mipsel-openwrt-linux-uclibc-gcc
and the compiler and linker flags for
CFLAGS = -O -DEXT94 -DPERMUTATE
LFLAGS = -x
And remove (or comment) this line:
@strip $(MAINFILE)
I hope you enjoy your Linux portable freedom, and write nice warriors to compete in The Hill. I will look into making a graphical version, although it looks quite hard in such a small window.

For Technorati: 4X5EHA6E4XYB

You may be also interested in