Since this blog is the nearest thing I have of a record of what I am doing, well I guess github might have parts of the story, but in general, if I want to recall what I did, here is where I come.

I started with a minor adventure of node.js land, including package managers (npm) and packages such as grunt. At one point I had my django site being served via grunt, which I undestand is another layer of the onion. Thinking this might be useful soon as I will be writing code. Live updates could be handy.

After this I still had to find some javascript with a pattern I could follow. By now I am getting a bit of a grasp on the parts. It feels like a really complicated way to render forms on a browser. But as that is something I've always found more painful that it should be, a bit more pain to find a better solution might be worth the trouble.

So Angular seed came to the rescue. I think in fact this is what Nick was using a few weeks back at the global hackathon.

So after a bit more fun of installing npm a few different ways and something called bower which seems to be package manager for angular.js.

But the best bit is that all this stuff actually did work. And I moved the app into my tree of code and that works under npm there too. So, I think I am 80% of the way to finishing my kindling.

So ten days to go until this project is supposed to be up and running. Now, this is a software project, just something I have been doing mostly for my own amusement, but it has a deadline and I am starting to hear that whooshing sound that deadlines make as the come by.

I am 70 days in, so what do I have?

I have a simple CRUD api built in django. It comes with data migrations and code generators that mean adding new models should require very little coding. I have the api documented using django rest swagger. And I can manipulate tables using the standard django admin interface.

I have also taken a look at django quiz and it looks like it will do a good job for me.

However, so far no proper front end. As far as the front end is concerned I am Dancing in the Dark

We had two hurricanes here in Bermuda back in October. It is winter now. Actually, a beautiful day today, 21C, sunny, few clouds and beautiful blue oceans. I just rode out of town on the bus. Must be one of the best bus rides in the world.

But, come evening, the sun disappears and a chill descends as the dampness comes out of the walls.

There is a lot of wood around the island, branches broken by the storms, whole trees knocked over. In many cases, all the branches have been cut off trees and the stumps set back up, green shoots of recovery are starting to sprout from these stumps.

Meanwhile, I have been gathering wood for fires most nights. It reminds me of happy days on scout camps way back when. Summer camps with the scouts were the best. We would cook all week on an open fire. Between us, and guided by Skip, we would figure out how to light fires, how to get a good fire going and then let it die down a little for cooking.

One thing you learn right from the start is, you can't start a fire without a spark.

And that is where kindling comes in. As I trim the bigger branches to get logs for the fires the smaller twigs drop to the floor. Once I have logs you can gather up a bundle of smaller twigs. This is the kindling. This is what lights the fire. The tiny pieces.

So, today I am setting out on a little kindling project. I want to add a little box to the Around the world in 80 days site with today's date and a couple of other labels showing 80 days from now and 80 days before today.

Now I could do this very quickly with django templates given the code I already have. But, I want to try angular.js. That way I will have a new tool I can play with. If it isn't working for me, then maybe it will end up like the bow-saw that I find less fun to use than the small hand axe, still useful, but not the tool I like to use most of the time. Or maybe, it will turn out to be a new and better hand axe.

So, time to light the blue touch paper and stand well back.

After a good break for the holidays, a couple of weeks in the frozen wastes of Canada complete with arriving home with a stinking cold I am now at the, "this code won't write itself" part of the 80 days project.

There is quite a bit to do, but I am still finding distractions to spend time on.

I have been using django rest framework to create a REST api for the project.

Creating serializers, api classes and url specs for these frameworks is largely a matter of generating standard boiler plate code. This is the sort of stuff I hate doing: tedious and error prone.

I originally wrote my own noddy code generator to help with some of this, but was less than satisfied with the result: it was fine for my immediate needs, but as I add more models to the application I would need to do more autogeneration. I had the inescapable feeling that my code generator was probably more trouble than it was worth. At the same time, writing code like this by hand is deeply unsatisfying.

So, just when it was looking like I would have to start learning angular.js and build the front end, Ned Batchelder came to the rescue on Twitter by posting a link to a new release of cog.

Ned described it as "a small tool that does a small job well".

Sure enough, it does. It lets you embed python code (disguised as comments) within your code to do the code generation.

There are a few features in cog that appealed to me:

• the code generator can be run in a way that over-writes the input file with the newly generated code
• it calculates checksums for the generated code and embeds them in the output. This is useful as it allows it to spot if you have edited the code since it was generated and hence refuse to re-generate that code.
• you can re-run and it will update the existing code, assuming the checksums have not changed.

Armed with cog, I came up with codegen.py to generate the boring bits of api.py as well as urls.py and serializers.py.

I am quite happy with the results. Doug Hellmann had suggested maybe cookiecutter might be a better solution to this problem. In my case, I am using cookiecutter to get the base files for my django application, but this code generation cannot take place until you add some models, hence the need for cog.

Better still would be to combine the two and add codegen.py and suitable urls.py, serializers.py and api.py to the cookiecutter template. I'll add that to the list of things to do, if only to delay the learning of angular.js ;)

I am lucky enough to live in a wonderful part of the world. Bermuda is another world. It really is.

A tiny island of just 60,000 people, a place with a complex history. There are a lot of wonderful people here. There are also some cool things starting to happen here such as code441 and wearebermuda.

It is Christmas time. The last Saturday before Christmas and there are parties everywhere. Many of them are company parties. You know the thing, you sort of have to go along. With luck you'll get to sit with some of the people at work you actually like. As the night goes on, if you are lucky, you will find some like-minded people to spend time with.

This year, no office party for me. So, we went to a local bar. There is a DJ every Saturday night at this place. The dance floor is always packed, for a simple reason: he knows his crowd. And he always walks around before he starts, plans his work and works his plan.

Last night there must have been 60 cars parked in the area. The place was packed. Everyone was having a good time.

Now only in Bermuda could a DJ start the evening with My God is Awesome, follow it up with Silent Night and already have his audience tapping their feet and getting into the mood.

I'd never heard My God is Awesome before. Actually, I am sure I must of heard it, just hadn't really listend. And I only really listened this time because it reminded me of the Awesome window manager I've been using awesome the last few weeks and it has had a huge impact on the way I use my computer. And I am only scratching the surface of what it can do.

So this was followed up with You can't hurry love, an old version by the Supremes. Another special song for me.

Then there was One Love, Electric Slide, complete with line dancing. Bermudian line dancing has to be seen to be appreciated. Last night was typical. A mixture of regulars to the place and those just there for the party. Everyone having a good time together. There were a lot of Happy people.

My music tastes are a bit weird. I pretty much follow the crowd. And my guitar music. I have been (very slowly) learning to play over the past few years. I have a few books with tabs, most of them have the word easy in their titles.

I would pick a song and try the patience of those around me by playing it over and over again, badly. I would look at the tabs of some and think, no way I can play that. But now I am starting to get a little better and a little more adventurous. I am noticing songs that have been there for years ready for me to play. Song of the moment is the The Joker. Like all good songs, there is lots of scope for changing the lyrics:

I'm a picker,
I'm a grinder,
I'm a lover,
I'm a sinner,
I play my music in the sun

Grinder not grinner, in tribute to the Sheffield Grinder.

And seeing as I have a bit of a git obsession:

I'm a puller,
I'm a merger,
I'm a midnight coder,
I get my checkouts from github.

Thanks Uncle, old stuff still works. Sometimes it works way better than the new stuff.

It is more than time that I get the project up and running on a host somewhere.

I could just check the code out on koding.com and run the project with django's build in server, but I feel I should run it under nginx.

I could have chosen apache, but I keep hearing people mention nginx so I want to see what it is all about.

Gooling nginx django koding threw up a guide to deploying django with nginx and gunicorn, so let's try that.

So all went well until I got to the bit about running the application with gunicorn.

My project layout is a little different to the one in the guide. In particular, the cookiecutter I used to create the project lays out the django config stuff a bit differently.

So, I did the following:

export DJANGO_SETTINGS_MODULE="80days.config.production"
gunicorn 80days.wsgi:application

This got me as far as being told:

ValueError: Couldn't setup configuration '80days.config.production.Production':
Secret value 'SECRET_KEY' is not set

Time to disappear down the SECRET_KEY rabbit hole.

So a little googling tells me a little about the secret key, but no advice on how to go about keeping it secret.

So looking at what the wonderful cookiecutter django created I see the following incantation in config/production.py:

from configurations import values

# SECRET KEY
SECRET_KEY = values.SecretValue()
# END SECRET KEY

So what is this configuration.values thing? Turns out it is coming from the django configurations application that the cookiecutter was kind enough to include.

OK, so it helps you not have to do

from foo import *

Now, that has to be a good thing :)

So reading the docs I come to:

django-configurations allows you to optionally reduce the amount
of validation and setup code in your settings.py by using Value
classes. They have the ability to handle values from the process
environment of your software (os.environ) and work well in
projects that follow the Twelve-Factor methodology.

I will leave this twelve factor thing for another day, but a quick glance suggests there are some good guidelines there.

So, digging into the docs for django configurations (which, by the way are quite excellent). I come to this little gem. In short, this magic values.SecretValue() is there to discourage me from actually putting my secret key in the config file.

This is good, because I am pretty sure I have committed this sin in pretty much every django project I have done up to now. Thank you cookiecutter :)

Having said that, I feel this another case where good intentions are likely to result in bad security, for more see my rant about computer (in)security

With a tight deadline, how many developers would just slap a value in the config file just to get things working and be done with it?

It seems that the more important something is to computer security the more byzantine and patchy the documentation will be at explaining how to make sure your computers actually are secure.

The problem is security is difficult. You only have to mess up once and all bets are off.

Invarialby, getting stuff up and running involves a lot of hacking around. Eventually, you get the thing working, but by then it is probably too late, even if you go back and plug all the holes And that assumes you can remember what you did along the way.

In the open source world products are generally configured in a secure way out of the box. You have to explicitly switch off things that are there for your protection.

That is all well and good, but without excellent documentation non-expert users (which in fact is every user a project will have: we are all novices when we start using a new product) are going to have problems getting the thing working.

The result is insecurity everywhere. And that is before we get into all the backdoors that various organisations have been more than happy to leave in products for their own purposes.

I don't have a good solution to this problem, the subject is complex. What is clear is that the current situation just is not working.

Following industry standards

To all those company's who are following industry standards you are a huge part of the problem. Those industry standards are horribly broken, possibly by design.

There is no substitute for hiring someone that actually understands the subject and listening to their recommendations. Blindly following standards that were introduced 30 years ago and which nobody can remember why they were introduced is lame.

It might get you off the hook with your boss: we are just doing what everyone else does, nobody got fired for choosing IBM, etc, etc. But the fact is that your security is really dependent on the fact that there are more attractive targets, such as companies that actually call themselves Target ;)

As an example, every place I have ever worked has a you must change your password every 90 days rule. Really? This rule was introduced based on the time it would take a good sized computer to crack a password given the hashed value that gets stored in password files... in the 1980's.

The idea being that if you change every 90 days it will be unlikely the crackers will crack your password before you change it.

NEWSFLASH: to all CIO's out there, computers have got faster since 1980. If you really want to protect against crackers that have your hashed passwords then you probably need all your employees to change their passwords about every five minutes.

Perhaps the one good thing that will come out of all the high profile breaches we are hearing about at the moment is that some of this archaic nonesense will be replaced by measures that genuinely increase security.

However, with the technologies that most companies are dependent on, the difficulty (or indeed near impossibility) of introducing genuine security without making it impossible for everyone to do their jobs, I think we are in for many more breaches in the coming years.

Indeed, I would wager that there have been lots more very significant security breaches, we just have not heard about them (yet).

I grew up in Sheffield in the UK. There are two football teams in Sheffield: Sheffield United and Sheffield United reserves. They play in red and white stripes and are known as the Mighty Blades.

There is also another team that goes by the name of Sheffield Wednesday, also known as The Owls, or ironically as The Massive, the owls or the Pigs. Curiously, the followers of The Massive also refer to followers of the Blades as Pigs.

If you grow up in Sheffield you end up following one of the sides. My parents grew up within walking distance to Bramall Lane. They were both Blades, as were my grand-parents.

Most families have members in the other camp, so for instance my sister married an Owl.

I now live in Bermuda and here the closest thing to the Blades/Owls rivalry is the Somerset v St George Cup Match rivalry. Most Bermudians don't get to choose who they follow: it is pretty much decided at birth. There should probably be a box to tick on Bermudian birth certificates to specify which side you are going to follow.

Both these rivalries are intense. There is always lots of banter and when the two teams meet if your side doesn't win then going in to work the next day is going to involve some ribbing.

Living away from Sheffield means when the Blades lose I at least don't have to listen to the fowls. In fact, on the rare occasions when I meet a Wednesday fan here it is good to meet a fellow Sheffield exile.

Now neither the Owls or the Blades have been all that good in my lifetime. Both have had spells in the top flight, both have had the odd cup run here and there, but both have had more dreary 0-0 draws on a cold damp February afternoon against some other struggling club such as Grimbsby or Crewe.

Many afternoons at Beautiful DownTown Bramall Lane (BDTBL) there has been nothing to cheer apart from maybe a stray dog wandering on to the pitch.

Back in the day there was no fancy digital scoreboard, just the letters A-Z with hooks alongside with room for someone to go and hang up the scores at half time. If you wanted to know which game was which you had to buy the program. But A was always for the Owls game.

What is more, the Owls score would get updated every time a goal went in. When the Blades play at home, the Owls are away. Since most seasons they are likely to be struggling the odds are they will lose.

So when the man wanders out and puts up the score and the crowd sees that they have let a goal in, the chant would go up:

Sheffield Wednesday Fucked It Up Again

On a good day, you would hear the chant 2-3 times.

And that is where swfiua comes from. In short, the loudest cheer at many a Blades game.

However, it is not all doom and gloom as a Blade. Last night was yet another memorable cup tie at BDTBL. We were drawn at home to Southampton in the quarter final of the League (Capital One) Cup, also known as the "whatever it is called this year cup".

At the time of the draw Southampton were defying gravity and placed 2nd in the Premier League. Meanwhile the blades were holding their own in League One, but not exactly scoring goals for fun.

In the meanwhile, the football gods have intervened, Southampton's last few league games have been tough ones and they have lost them all.

All was set for a classic cup upset at BDTBL on a December night in Sheffield.

There was a good crowd. Night games at BDTBL are always special.

At this point it is worth saying a word about Sir Nigel Clough.

It is difficult to talk about Nigel without mentioning his father, although I hesitate to do this, it cannot be easy being Brian's son.

Brian Clough was an absolute genius of a football manager. He was always entertaining in interviews, always had sides that played good football and his teams always respected referees and won fairly. Oh, and Brian always took the cups seriously.

Nigel Clough has now been in charge for 18 cup ties, all whilst the Blades have been down in the third tier of UK football. They have only lost 2 ties.

Last night was a wonderful win. They took the game to Southampton. The longer it went on the more I started to believe they might actually win. As a Blade, this is an uncomfortable feeling. So many times, just when you think we are going to do something it all unravels. The journey is always fun though, but we all know how it ends.

Well last night was one of those rare occasions when everything went well. We got to half time 0-0. We were the better side first half.

The first 10 minutes after half time were going to be key. Southampton no doubt got a talking to, we had to be ready for them to come at us. Southamptin did come out a little stronger, but the Blades held their own.

And then the unbelievable happended, one of those freaky goals, ball trickling over the line, with keeper and Sparky McNulty both racing to get to it. Sparky won the race and sparked off a fantastic last 30 minutes.

We know it will end in tears, but it is nights like this that make it all worth while.

As I said to a Chelsea fan (who also had a soft spot for Derby) before the game, the great thing about following the Blades is if we win two games on the trot it is like winning the Champion's League.

So on to the semis, looking forward to the draw later today.

Oh, and those wondering how the Owls did this year in the League Cup: they lost 7-0 to Man City in the third round. swfiua * 7.

I am now 33 days into the project and anyone reading this blog ought to be wondering how it is actually going.

One thing I am enjoying about this project is there is no project manager. Well I guess, technically, that is my job.

I have been writing code long enough to see more than my share of magic bullets that will make everything go smoothly. Sometimes these were formal approaches to coding and project management. Other times the bullets are coming from the open source community: ways of working that people have developed for themselves.

In general, the open source stuff speaks for itself. Nobody is mandating that the projects work the way they do (actually, in most cases the BDFL is probably mandating the standards). But the point is that the standards are being driven by those that know the job best.

Around the time Extreme Programming became popular I encountered the following:

Time, resources, scope and quality:  pick any three

Fix three of these and then you have to let those doing the work tell you what the fourth will likely be. I find this to be a very useful tool. Letting others set all four is a recipe for failure.

In the 80 days project I am fortunate. Time is set to 80 days, but I pretty much have control over the other three.

Actually, that is not quite true. As far as scope is concerned I will need to deliver something by February 1st and actually need somewhere people can register for the competition by January 1st (give or take a few days). But overall, the scope will be whatever gets done in the time available.

Resources are to some extent under my control. Right now there is only me working on the project. Actually, that is not really true. There is a whole army of open source heroes working on this project, they just don't know they are working on it.

So far I really have not written much code. However, I do have a site that has a pretty landing page, and about page and you can register either by setting up yet another user id and password or by using your google ID and open auth.

I didn't have to write any code bar a tiny bit of html and a fair bit of configuration of django. Mostly it has just been a matter of finding packages, reading how they work, follow the instructions.

To some extent, this whole project is turning into an exercise in how little code I actually have to write. Code re-use is working well for now.

We had a friend over the other evening. I have a home full of half working tech. Stuff that really could do something, given enough time.

A lot of my techy stuff has been neglected for a few years, so it is suffering both bit and hardware rot. So a good part of last night was spent exploring why stuff wasn't working, in between finding a few gems.

So today, I had a few things I wanted to look into:

• why wasn't the server with the music on responding,
• installing a fedora iso onto a usb and seeing what it is like.

That server with the music; well that thing won't even boot off a USB key; though you can change a bucket load of things in the BIOS. And at one point it did seem like it would boot over the network.

Next stop, fedora install guide kindly gives the incantation:

dd if=fedora*.iso of=/dev/sdx  (x in my case was b)

So I booted up a laptop (the one with the battery that lasts 30 seconds) off the USB and fedora booted. I did not have time to have a good look around, but it seemed snappier and easier to use than the Ubuntu desktop on the machine, more like the awesome wm experience. I will definitely take a nother look at this.

Back to the music. I resurrected an old mythtv box and tried to put fedora on it. I couldn't get it to install on the old myth box. It may have had something to do with some network issues that were going on at the time. Also possible there was some issue with this boxes USB, it was an old machine, so might have old firmware. Or the USB isn't an innocent little USB. But I digress.

So, I tried arch. This was like being beamed back to 1997. It was fun for a while, but then I typed the wrong thing and lost the internet connection, which was what I needed to get the software to fix the internet connection. I did Ctrl-D out of the chroot and if I'd been awake when I booted back into it I might have skipped the second install. I gave up at the bootloader bit. I know, I was nearly there :(

I might have to go back to arch one day when I have lots of time on my hands.

So more rabbit holes. Eventually, led to Debian. By now the dining table is buried under cables. The IP over powerline is up and running again and 2-3 boxes are in the pile marked broken.

The Debian install was been a joy, apart from the sporadic (or rather repeated) interruptions to anything needing internet bandwidth (a problem anywhere with expensive or unreliable bandwidth).

I suspect I am biassed because over the years I have spent more time with Debian based systems. First with debian and then indirectly via Ubuntu.

I liked Ubuntu primarily for its python packaging. Ubuntu still is a fantastic distribution for python. As an interface for the sort of programming I like to do it is becoming less responsive, or at least that is my feeling. Also, I am feeling myself further removed from the hardware and the software configuration.

At the same time, python packaging has come a long way. With PyPi and pip install the linux distro no longer matters so much.

Debian doesn't seem to have changed much in philosophy. Most of what I learnt back in the day still applies. It led me through the install with just enough information to make an informed decision about the questions. It guided pretty well, hinting which way I should go.

As of now I am yet to actually get it serving up the music, so this will most likely not be the last distro related posting.

Last weekend I took part in the global hackathon run by koding.com as part of Team Bermuda aka Three ace-boys and an ace-girl.

The other members of the team were:

• Nick Hoskins, the one who told us all about it
• Beckett Simmons, residing koding.com expert
• Veronica Windus, working remotely from the frozen wastes of Canada

The idea was to create a web based application in 48 hours. There were five themes to choose from, we chose:

No one reads the fine print (ie TOS, EULA, legal documents) anymore
yet every site has them. Devise a creative/interactive solution.

So, before we could start we needed a name.

Since the average EULA is about as easy to read as the classic, James Joyce' Ulysees we took the liberty of borrowing the name, and then abusing it and came up with, Eulasees.

There is a tenuous link with Bermuda in that many songs of Thomas Moore who lived in Bermuda for a while, are cited in the works of James Joyce.

So, back to the hackathon. This nominally started at 4am Bermuda time on Saturday 6th December. Team Bermuda hence decided to start work at 10am, gathering at the Watford Sports Club.

In fact, things got off to a slow start due to several factors:

• lack of coffee.

  Actually, there was coffee, but we only had a tiny bodum to make it in.

• A fellow Blade turned up at the sports club.

  These days, blades are rare as hen's teeth, so progress came to a complete standstill whilst yours truly reminisced about the good old days when the Mighty Blades were in the heady heights of the first division (technically, they are still in the first division, it is just that now first really means third, English footy uses a weird number system).

• At least one of us working on a laptop that was missing lots of the standard development packages.

• Flaky wifi.

• No real idea what we were going to do.

This was very much a make it up as you go along event. We talked about a few ideas, delegated lots of stuff to our ace-girl, who was presumably still digging out from under a snow storm.

The rough plan was a to create a restful API with django and build an angular.js front end.

The idea was to load EULA's into the system, allow users to highlight areas of text and apply tags to these areas.

These tags would have a short name indicating what the contract clause really meant and a longer text body explaining what it was all about really.

Once we had this and some data in the system the plan was to create some views allowing users to see which EULA's had which clauses and other such fun stuff. One idea was to add a quiz, getting people to guess whose EULA had which clause, possibly using django quiz.

For most of Saturday we pretty much ignored the koding.com virtual machine, we were having enough other problems.

Actually, the angular stuff was going quite well.

The django side was going a bit slower. We were using cookiecutter to create all the base files for our django apps and project. This works really well, but does come with a lot of extra goodness that we probably did not need at this point.

pip install kept failing on obscure compile errors due to missing -dev packages. I love pip, but with a big requirements list and a dodgy wifi connection the downloads were taking way too long and I couldn't get it to cache things reliably. More on this later.

Having said that, being able to do:

python setup.py register

Followed by:

python setup.py publish

To make the django app available on PyPi (and hence pip install magic just works), was very cool.

Things went well on the Saturday. Mid-evening we stopped for food and more coffee (by now Nick had brought a decent sized bodum down from his house, so the coffee shortage was averted).

We decided to work until we were too tired, then get some sleep ready to code until 4am on Monday morning.

It is all a bit of a blur now, but at some point we decided we should see if any of our stuff actually works on koding.com.

I am sure the koding.com IDE is a joy to use, but now was not the time to learn it. Fortunately, Beckett had given us the number one pro-tip for using koding.com:

Getting ssh to koding working is well worth the effort

Further, it didn't involve anything more than adding an ssh key to the ~/.ssh/.authorized_keys file.

Suddenly pip install was flying along. Clearly, we should have been doing everything on the koding machine.

Having noted how well the koding VM was working, of course, we continued to work locally.

We were, in fact, so effective at this that it was midnight on Sunday before we decided it would be a good idea if the angular front end could talk to the django rest API.

No prizes for guessing the answer to this: hell no.

Angular was either failing completely to connect to the django API giving obscure messages about cross-site request forgery

At this point we probably should have left the angular front end as an example of where we were heading and just fed it some toy data.

Instead we learnt way more about csrf's than we ever wanted to know, in between cursing django for its attentiveness to security.

After much thrashing around django cors came to the rescue. By now, however, it was past 3am and Nick was having to use a new function to make the restful API calls.

We got the two pieces talking, but ran out of time to fix up enough of the angular code to get a viable front end.

The project code is on github, as eulasees, with the the django eulasees app in a separate repository.

You can also view the docs for the eulasees API that was created. These are created with the amazing django rest swagger and allow you to try out the API.

We did manage to load some data into the system, with the django admin system coming to the rescue as the user interface of last resort.

We should probably apologise to google, since their terms of service was the toy example we decided to use. You can actually get all the data that we entered with this one API call. So, if there are any google lawyers reading: we love you really, we know you are not evil. Just one question: what is with the paragraphs IN ALL CAPS? ARE WE BACK IN THE 90's ON A SHOUTY FORUM?

Remarkably, we got an email yesterday informing us we made it through to the next round of the competition. We can only assume that pretty much everyone that got any sort of site up got through the first round. You can even VOTE FOR US :)