Category Archives: Projects

My Projects

Arrival at Zurich – Wikimedia Hackathon 2014

Posted on by

I’ve arrived!

After some hiccups with API, and my shoes setting off the metal detector causing me to have a full body scan et al. by the security guards, the flight here was relatively uneventful thankfully. SwissAir provided free drinks and lemon cake inflight, which was unexpected, but welcome.

The sight of the Alps as the plane was coming in to land in Zurich reminded me how much I love being in the mountains. Although Zurich is not really in the Alps, it’s not too far away, and just the overall feel of Switzerland (France too, but I’m not in France) is really relaxing and calming.

I’m quite glad to be back on the continent again, it’s just a completely different atmosphere. Unfortunately, I’m not going to be able to get out and explore Zurich much while I’m here, as I have a lot of coding to do for the Wikimedia Hackathon!

It’s good to see faces again I’ve not seen for a year – and also to put faces to names I didn’t meet last year in Amsterdam. While the Hackathon hasn’t officially started yet, a fair few people are already getting down to work, including myself. I’ve been making sure that the code I’m going to be working on is in a fit state to have new features added to it, so making sure I’m not building new features on top of existing bugs. I’m also attempting to replicate my working copy of the code that’s sat on my desktop machine onto my laptop – I thought I had all the pieces I needed, then I remembered how long it has been since I last did software development on my laptop!

Power initially proved to be problematic too, while I thought I had a Swiss power adaptor, all of the adaptors I had were in fact only European ones. A mad dash around the shops before I left proved fruitless, so I had to buy one in the airport. UK and EU plugs being larger than Swiss plugs, and with the elegance of the Swiss sockets allowing three plugs to be inserted in a space only slightly larger than the UK and EU plugs means that when an adaptor is plugged directly into a Swiss socket, the ability to use the other two Swiss sockets is severely limited. Thankfully, a fair few people (myself included) have brought extension leads (mainly European), which others can plug into without problems.

The main aims for my time here are to get a framework for supporting OAuth within the account creation tool, allowing us to replace a few key pieces of functionality for which the implementation is not ideal. Firstly, I want to replace our confirmation of on-wiki username, then work towards implementing the welcomer bot to use the account creator’s actual account, rather than faking their identity with a bot. Lastly, if we can allow the account creation tool to actually create the accounts itself too, so we don’t need to redirect to the creation web form, it would be awesome.

I’m hoping to get through as much of that list as possible, with assistance from the development team of the extension, but overall I’m aiming to get far enough with a working framework so that I can complete the rest without assistance. If I can also complete some of the outstanding issues that the extension has too, and learn more about MediaWiki in the process, it will be even better.

Wikimedia Hackathon 2014 – Zurich, Switzerland

Posted on by

In a few weeks time, I’m going to be heading to the Wikimedia Hackathon in Zurich, Switzerland.

I’ve been granted a scholarship by Wikimedia UK to cover my travel costs and accommodation, so I can spend a long weekend hacking on code for Wikipedia-related things in the company of a hundred or so other developers from around the world.

I’m heading there to work on integrating the Account Creation tool with the Wikimedia login system so we can replace our broken welcome bot with posts from the creator an account. In the past we have faked it by making the bot sign as a different user, but we’re hoping to allow the bot to edit as other users using the new OAuth tools.

I’m also hoping to learn a lot more about the internals of MediaWiki, the software which powers sites like Wikipedia, in the hope that I can get a lot more involved in the development of this remarkable piece of software, both the core application itself, and extensions. I’ve already done a bit of work on extensions, but I’ve hardly done anything with core. I’d love to learn more to be able to get much more involved with it.

One of the other things I’m looking forward to is the OpenPGP/GnuPG key signing party that’s been suggested, where people can get together and verify each other’s identities, then go away and sign their keys as being valid. It’ll be the first time that I’ve ever been to something like this, and it will be good to get a few more signatures on my key!

It’s going to be really good to meet people, learn, and importantly hack on code to try and get something worthwhile done!

Raspberry Pi, and Raspberry Pie

Posted on by

:(

I went to bed early last night cos I knew I had to be up early this morning. Getting up at 05:30 is something that is incredibly rare for me. Unfortunately, it was a bit pointless this time around.

I was trying to get one of these:

Copyright 2011-2012, Raspberry Pi Foundation

It’s called a Raspberry Pi, and it’s essentially a computer the size of a credit card. With an ARM11 processor manufactured by Broadcom, 256M RAM, HDMI/RCA video output, 2 USB ports and Ethernet, powered by micro-USB, with SD for storage. What more could you want for a basic computer system?

Initially, it’ll run a variant of either Debian or Fedora Linux, and my plan is to use it either as a local web server/proxy server/dns server, or maybe running XBMC as a low-power media frontend. It’ll also be good for simply messing about with.

Oh, did I mention the price of this thing? No?

$35 (USD).

In the process of trying to order one, we (everyone who wanted one) brought down not only RS Components’ website, but Premier Farnell too in mere seconds. They were warned about the traffic, but seemingly thought the warning a bit frivolous since both of them didn’t adequately prepare for a HUGE traffic surge.

And now I have a craving for raspberry pie.

Progress reports really do slow things down!

Posted on by

This is something I’ve known for quite a long time, but if you’re ever thinking “it’s not that much extra power to add progress reports”, just make a mental note of exactly how many progress reports you’re making, and try commenting them all out.

BeebMaze generated a new maze in mere seconds, but at the same time, that was seconds too long. As part of refactoring, I took out the progress reports with the intent of putting them back in, since there wasn’t that many.

It now generates a new maze in about the time it takes to paint the screen once. :D

(Download this build from the build server now: Build #18)

Regular Expressions Tester Engine

Posted on by

I’ve actually taken a bit of time to fix up, make user friendly, and publish a tool that I wrote a little while ago, and has been very useful to me.

The regular expressions tester engine basically provides a front-end to the Microsoft .NET Framework Regular Expressions engine, allowing you to test regular expressions you’ve written to your hearts content.

As usual, I’m releasing it as-is under the MIT licence, and as usual the source can be found on GitHub.

For a page which might have more information on it at some distant point in the future, but more importantly has a download and bugtracker link, see this page on my main website.

ACCBot’s recent breakage

Posted on by

A couple of weeks ago, the IRC bot that we use over at Wikipedia’s Account Creation Assistance project decided it would stop giving notifications to the IRC channel. Previously, it used UDP as the transport between the web interface and the IRC bot. However, for some unknown reason, this stopped working.

After seemingly endless messing around with PHP, netcat, more PHP and a bit of telnet, I came to the conclusion that it was fucked, and there was no way to recover it with any ease.

Previously, the code looked something like this:

For receiving notifications over UDP, that was all we needed – it worked and was semi-secure. However, when it stopped working, I took a more radical approach.

You’ve probably heard of Amazon Web Services (AWS) by now, if you haven’t then I recommend you take a look.

One of the AWS services is something called the Simple Notification Service, which seems to be exactly what I want – a notification system. However, the only notification endpoints are HTTP pings, e-mail, or an SQS endpoint.

SQS is what I chose eventually – it’s another of Amazon’s services, the Simple Queue Service. This has the “advantage” of queuing all the notifications so if you’re offline you will still get them all. However, for our case this isn’t ideal, but the bot isn’t usually down for long if it goes down. So, I decided to go for SNS->SQS over HTTPS as the transport for the notifications, rather than UDP.

Of course, code needed changing – at first I thought drastically, but it turned out to be a much smaller change than I anticipated:

It looks small, just another explicit check to see if we actually received anything. That’s until you realise that I wrote another function to take some of the work off to one side.

There’s not much that’s changed, but it was an interesting technical challenge :P The only thing that has noticeably changed is the lag from notification generation to display on IRC – can be anywhere up to about 5s if you’re unlucky!

Strobe Light

Posted on by

Growing tired of the poor strobe light applications available on Android Market, I decided to build my own.

Lots of the so-called strobe light apps which already exist on Market seem to be of the type which use the phone screen as the “strobe” – this isn’t good cos it takes a while to update the screen, and it isn’t as bright as the camera flash led.

My app is a very simple one – a quick loop to set the flash on, wait a while, set the flash off, and wait a while.

The two waits are configurable with sliders, with values of anything from 0 (theoretically) to 500ms.

The only problem is the fact that developing camera-based applications on Android is a bit of a dark art – not all manufacturers seem to use the Camera API to access their camera. Therefore, camera-based apps can be a bit hit-and-miss as to whether or not they will work. As I only have the HTC Desire, I can’t test it on anything other than the Android Emulators, and my actual phone. I have, however, had reports of it working on a number of devices, including the Samsung Galaxy S2, HTC Incredible, and the HTC Evo 4G. There have also been reports of it NOT working on the Galaxy Tab, Droid X, and the XT720 Milestone.

As all the flashing takes place in an infinite loop, there’s no time to handle GUI events. Therefore I branch out the flashing bit to a separate thread, and use Handlers to make the necessary calls back to the main thread to update the GUI as necessary.

The main reason to update the GUI is to make sure that should the strobe thread get stopped in any way, such as on activity changes or screen re-orients (currently disabled) etc, the GUI “enable” toggle button actually shows the correct status, as well as to easily display any error messages as necessary.

There’s still a few outstanding bugs (screen orientation!), and there’s a few changes I’d love to make to it, but it’s coming along quite nicely, I just need more time to actually work on it!

Technology updates

Posted on by

Quite a bit has happened recently!

I’ll try and give a bit of info about each, there’s quite a lot to write about though!

EyeInTheSky

An IRC bot which stalks changes to Wikipedia based on regex matches. Read more

caterpie beedrill as minecraft server

Those of you who use my Minecraft server will know that the server known as “caterpie” was bad. Any more than one person connected, and it would lag to the depths of hell. I’ve replaced it with a 1.7G beast of a server, still running as an Amazon server instance.

stwalkerster.co.uk

I registered a new domain! I’m still in the process of setting the thing up though, but that will become my main website, not quite sure what I’m gonna put there yet, but it’ll probably replace what webspace I had at http://helpmebot.org.uk/~stwalkerster/

ACC‘s IRC Bot, ACCBot

A couple of weeks ago, the IRC bot that we use over at Wikipedia’s Account Creation Assistance project decided it would stop giving notifications to the IRC channel.

I’ll write more about this at a later time, hopefully soon, but probably when I’ve finished messing with my new site :P

strobe light

Growing tired of the poor strobe light applications available on Android Market, I decided to build my own.

I’ll also write more about this too at a later time – I think this bit and the last one need their own posts.

EyeInTheSky

Posted on by

EyeInTheSky is one of my newer projects, an idea which I’ve stolen from two other people.

Wikipedia has so many modifications being made to it that it’s just not possible to keep an eye on everything you want to watch. While the MediaWiki software has a feature known as a watchlist, it’s neither flexible nor easy to use in my opinion.

EyeInTheSky is an IRC bot (seems to be my speciality!) which monitors the Wikimedia IRC recent changes feed, compares every entry to a set of regular expressions and reports them to a different network.

It’s possible to set up the bot with an entire XML tree of regular expressions matching on the username, edit summary, and page title. There are also logical constructs which allow more-or-less unlimited regexes to specify what exactly you want to watch.

For example, with this tree, I could specify I wanted to stalk all the edits which are made by someone with “the” in their name, “and” or “or” but not “xor” in the page title, and with “train” in the edit summary:

I also can set a flag, something that I can then set my IRC client to respond to, and it will speak that flag for every stalked edit.

Of course, it’s not just edits that can be stalked this way – log entries are sent to the IRC RC feed in the exact same way. It’s just a case of specifying Special:Log/delete as the page title to get the deletion log, for example. The entire log entry except for the time/user is sent in the edit summary field. This means the same system can be used to stalk log entries as well.

The bot logs all stalked edits, and is capable of emailing the entire log to me, so I can clear the log when I disconnect from IRC, and when I get back on, I can email the log, go through what I’ve missed, and catch up.

I’m planning on making it multi-channel too, with probably multiple people able to command it to email them the log. I can already tell it to not email certain stalks, especially as some of the stalks that have been set up are not things that interest me, but rather interest other people. I just ignore those when it reports them, and have it set not to email me for those stalks.

There’s quite a lot this little bot can do, if you want to learn more, I’d recommend taking a look over the source code, and see what you think!

The source code is available on GitHub here.