plugins and packages and extensions and bundles, oh my!

I’ve had a gripe about open-source (okay, free software!) package management for the longest time.

I would give a kidney (I swear I nearly would) if my OS’s package manager managed all my packages.

But it does you say?

No. I just spent the last 10 minutes installing a plugin (Vim-Ruby) for Vim. First I decided which Vim package manager I should use. I chose Pathogen fairly arbitrarily. Then I checked whether Ubuntu itself packaged Pathogen. Yay, it does. Then I checked whether the Ubuntu version is fairly up to date. Yay, it is. Then I installed Pathogen via Synaptic hoping that it will stay current. This is not always the case. Ruby gems (and their attendant package managers (yes, plural!) Rubygems and Bundler) are a big not always the case. Then I install Vim-Ruby like so:

git clone git:// ~/.vim/bundle/vim-ruby

And then I have to remember to keep this up to date independently of apt.

Why, can’t sub package manager’s hook into the system package manager? I want to be able to go to one place to install a blob of code, be it a system level package, a language level package, an editor level package, a browser level package. Last I checked:

PHP             Pear
Ruby            Rubygems, (mention not RVM...)
Python          Eggs
Haskell         Cabal
Chrome          Extensions
Firefox         Add-ons
Eclipse         Aaaaaaargh
Vim             Pathogen
Emacs           ELPA (? I don't know, I'm a Vim guy)
Perl            CPAN
LaTeX           CTAN

There has to be a better way people. Even if it’s just exposing each plugin/package/extension sub-system to a central registry so that they appear centralised in Synaptic. I think I may have blogged about this before. In a fevered dream.

I’d be grateful for any corrections and more subsystems to add to the list, probably should make a wiki.

Don’t get me wrong, I appreciate the enormous coordination problem. But if we can do it at the OS level, can we not do it at the subsystem level. I guess it’s mostly programmers that’ll encounter this, or maybe it isn’t? There must be a better way.

The Different Types of Open

I used to think open beat closed and that was that. Linux goodWindows bad. Open better. Closed worse. One type of open. Simples.

A neat and easy dichotomy, open versus closed source, one type of open, what’s not to get?

Take the word better and apply the is/ought distinction to it. Not that simple it turns out. The free software movement is predicated on the notion that proprietary software is anti-social and, to put it bluntly, immoral – it is worse in the ethical sense. The open source camp take a more pragmatic approach – they say that proprietary software is a worse way engineering-wise to build software. I won’t go into the fine details here, I’ve done so elsewhere. But you get the idea.

Now the free software foundation defines software to be free (open if you will) if the software adheres to four freedoms

  • The freedom to run the program as you wish, for any purpose (freedom 0).
  • The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
  • The freedom to redistribute copies so you can help your neighbor (freedom 2).
  • The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.

Mostly this gets boiled down to having access to the source code. The source code is open to you. For now let’s just pretend that these finer distinctions don’t matter.

For the longest while I thought that this was the openness that mattered. Initially it came from the desire to being able to tinker (pragmatic) but then once I saw how proprietary software could lead to abuse I began to take a more moral stance. But I’m not a zealot, I run Ubuntu which has non-free bits and have run MacOSX in the past and will use Windows in the university. But generally speaking I have drunk the FOSS kool-aid.

But actually, in the first place, for the longest while I had no conception of this distinction. It was something I luckily managed to learn. Partly from being a dilettante developer and tinkerer. Partly fortune. Most people never come to conceive of the divided state of affairs in software in this manner.

It was with the release of iOS and the iPhone and the Apple Store that I realised that there is for sure more than one type of open. Say I develop an app for the iPhone. I need to pay Apple money and get their permission and abide by their rules to get my app in front of users. Wow. I don’t think this was ever in Microsoft’s DNA. That is why proprietary (rather than closed) software is a better moniker for software where one does not have access to the source. Because DOS and Windows have never precluded 3rd party developers from their platform in that way (give or take secret APIs and breaking the software of direct competitors: *cough* DrDOS *cough*by way of example). So in a way DOS back in the day and Windows were open to some extent and so calling them closed misses the mark, proprietary (or non-free) is more apt. Hence the PC revolution. Nowadays Android allows what is called side-loading. Android is not only more open because the source code is not proprietary but it is more open because you can side-load. MacOSX has always allowed this so Apple has made its mobile offerings more restrictive than it s desktop and server offerings (because they are aware that the paradigms are seen as different and there would be a geek revolt). Thems are nice handcuffs Apple. So this is kind of freedom 0, being able to get the bloody app onto your device in a fairly hassle-free manner. Jail-breaking does not count. Let’s call this open loading.

Now Microsoft has long made its document files opaque binary blobs. Let’s call this open formats. Software is data and code. Data gives us format tussles. Code gives us source tussles. And what about opaque binary network protocols? Let’s call this open wiring. Again, the internet is the internet because the wiring is open from the bottom of the stack to the top. As opposed to Netware, say, or SMB.

I think that it is safe to say that a lot of battles have been won in relation to open source, loading, formats and wiring. Let’s call these collectively open platforms.


The free software movement started way before the web was a twinkle in Tim Berners-Lee’s eyes. (That reads badly, never mind – this is off the cuff.) I want to talk about two other types of open that become apparent in a radically networked world. The first is the openness that federation brings. The second is the openness that Karl Popper referred to in The Open Society and its Enemies.

Let me make this clear. The postal service is open because it is federated. The plain old telephone system is open because it is federated. Email is open because it is federated. Internet relay chat (Irc) is open because it is federated. Don’t get me wrong. I have a fixed line or mobile telephone provider but I can switch if I’m unhappy and the person I’m calling does not need to have the same provider. By this way of thinking Facebook, Twitter, Google+, Skype, Viber, WhatsApp, Line, Facebook Messenger, Instagram, Snapchat, ICQ, Apple iMessage, Microsoft WhateverItIsCalledIDoNotActuallyKnow, and on and on, I’ve probably forgotten a whole bunch, are not open. We can clearly see from this that social networking and instant messaging are not open. Let’s call this open social.

I think that it is safe to say that the defenders of openness haven’t even gotten to this battlefield yet. This is an exaggeration, sure – a rhetorical flourish. There is Diaspora* and the Freedom Box and SIP and Jabber and so on but we’re a long long way from the Android of open social.

Let’s call open platforms and open social together open ecosystems. To my mind if you use Linux and email (and irc) you’re more or less using an open ecosystem but you don’t have the key parts of social like following and un-following and rating and sharing and so on. What social networking and instant messaging desperately needs is radical federation and it can’t come soon enough for this despairing digital voice.

Since Snowden began revealing what “intelligence” agencies do in the dark we have been getting a steady drip-drip of troubling revelation after revelation. It would in truth be just too depressing to even begin to enumerate the ways in which our notions of our digital lives as private have been eviscerated. (My editor promises me that that sentence will be put through the wringer.) Government (or the State) argues that there must be a certain amount of secrecy for it to function effectively. I think the defining political challenge of our time in the era of digital technology is to figure out how transparent government needs to be.

When we have open ecosystems and open government then we’ll have an open society. I think while we’re waiting for open government we need open ecosystems plus pervasive and strong and secure encryption. I believe Popper argued that only open societies can do science. I think that claim has been falsified by all actually existing nation states. I think science happens despite the lack of openness in society. I don’t think we’ll get open societies without open ecosystems and ubiquitous and rock-solid encryption. It would be interesting if we were able to perform sociological simulations that could give us a blueprint for which types of societies nurture openness best. I know that sounds like total pie in the sky but maybe we’ll be able to perform such magic sooner rather than later.

Does anyone out there have any other types of open they’d like to share?

(1st draft, getting it out there, pointers to typos and brain-farts gladly accepted)

Mathematical tools in a digital world

Greetings. Apologies for the delay between postings. I spent the summer revising my master’s thesis. It is called “Mathematical tools in a digital world” and it is archived on the university of Jyväskylä’s library website. The permanent web address for a brief description and download link is right here

I created it in LaTeX and have exported it to PDF. That is the format that is available from the archive and the download link is this

In truth the work did not take the shape I imagined it would but never mind. There are three main parts: one is an overview of copyright and copyleft, two is a description of mathematical tools in general and a survey of one open-source project — Sage — and its proprietary competitors, three is a hands-on look at how computers and the digital are affecting mathematics.

One. I talk about the origin of copyright in England and the original intention of the law. I am indebted to the great work of Lewis Hyde and Common as Air. From there I show how the free software movement (richard Stallman and the GPL) co-opted the idea of copyright to undergird and make enforceable free software licences. There’s a lot more meat in there but that gives you the general idea. If anyone knows of a brief pamphlet-sized text that provides a similar run through let me know.

Two. This was the original impetus for the work. I wanted to figure out why open-source mathematical tools have not reached the level (this depends on one’s criteria of course) of proprietary tools. The tool — Sage — seemed most advanced so I decided to compare Sage with its competitors. In the end it became obvious that for such a complex piece of software this is not practical so I designed a questionnaire to look at the “human interest” angle of people involved. The results are collected in the thesis.

Three. This section was in a way the most fun. I look at the implications for mathematics and mathematical tools as a consequence of the move to the digital realm. Computer-assisted proofs, advanced visualisations, not simply a speeding up of calculations but an opening up of previously unreachable vistas. Using LaTeX I was able to embed fractal imagery into the thesis itself putting the thesis itself out of reach of the pre-digital world. I also have explored how computational mathematics is affecting the production of art.

Granted, the whole lot is a bit of a mixed bag but I think lurking within this thesis is an approach to the digital arts and humanities that has seldom been taken: from the code up.

I am linking to it here not to glorify myself but as with my undergrad work in the hope that someone out there finds it of interest.

With that I bid you good day. Good day!