Friday, July 18, 2014

The Next Big Thing - A Small Whinge Expanded Upon

So after my last blog post, the one person who I know still reads this blog asked me a question... why does it matter?

The answer is something that I've seen in really BIG organisations. So I figured I should describe why it matters...

Imagine you're running a big organisation. Your computer systems are vital to the way that work. Computer down time isn't just me stomping around getting pissy about things not working. It's money. When I was working for one of those larger organisations, we were advised that it was our responsibility to have "Public Liability Insurance" to the tune of $1,000,000. That's because if through our actions we caused a problem that resulted in a lack of production, we would be held liable. It was a BIG organisation and so an hour or two could result in millions of dollars in lost production.

What this teaches you is that you ALWAYS have a plan. You should be able to go back to a state before the change. And here's the problem with APT. It's not, in any way whatsoever, enterprise grade. I could not, in good conscience, use it in that sort of environment.

Granted, my position in this organisation meant that I was often on the back foot. I was looking out for the changes to make the desktop software work. It was on me to make sure that part of it worked, the information got back to the software packaging team and the updates rolled out as quickly as humanly possible. React, react, react. I LOVED IT!

But that's beside the point. So the problem and why it's a problem. Imagine you have a package. We'll call it vital-software.

  • The initial version of vital software is 1.0.
  • All the clients have, in the lifetime of their desktops, upgraded to 1.5 without any problems.
  • No security updates have been issued.
  • A new version, 1.6, is available - but it's got a great big giant problem that means that vital-software is no longer working the way that it should... but it's absolutely vital.
Which means vital-software NEEDS to be restored to a working state ASAP.

With the way that apt currently works, apt knows about versions 1.0 and 1.6 only. So once upgraded to 1.6, you can't go back to 1.5. You can only go back to 1.0. What if file formats have changed in that time? 1.0 doesn't work either...

In reality, you'd have a small number of testing machines to mitigate your risk. One of those risks was me... if I didn't know what I was looking for in terms of functionality from an application, I had no idea whether something was wrong. A test lab was the only thing that saved me. It turned out that the software involved would have brought down a call centre... In the end the application had to be "ring fenced" (pinned in Linux terms) due to it having a requirement of an older version of mscomctl.dll. Luckily I had the vendor helping me with this one...

And it opens up opportunities... functionality changes in more recent versions (because the software is updated in the next release anyway... why couldn't you run it on an older release?).

I've been thinking about it and I reckon I need to fork the apt stack... call it RAPT (Really Advanced Packaging Tool) and hopefully have it merged back into APT (because forks are useful but generally should have different aims and/or ideologies behind them for them to be of any value)... Or risk producing my own distribution that leverages Ubuntu packages... In which case, it would have to be a rolling release...

Monday, July 14, 2014

The Next Thing - A Small Whinge

I've started work on what, in my head, is the next big thing.

Basically, what would it look like if the tools existed for Manaiakalani to do what I did for it?... I mean, what would it look like if they didn't have me? How should it look? How could they do.... well... everything?

While I'm working on this though, I'm wondering about the apt system. I mean, it's good. Or rather, it was great for what it was designed for, back in the day... But what improvements has it gone through recently?

I'm not comparing this to anything. I have no idea how yum or similar systems behave. But apt, which I've been looking at, kind of sucks.


When apt updates, it downloads a package file. And that package file is flat. It contains only one reference to a package. What does this mean? You can have a maximum of 3 versions (not strictly true - you could define a whole new "distribution" (I'm putting distribution in quotes here as I don't mean "distributions" as in lucid, karmic etc. but rather distribution channels (security, updates, main).

Anyway... it's a mess. We have the standards to be able to do much more descriptive package files so that a system has more options for versions of a particular piece of software... We have the technology. XML would do. Something along the lines of:


In which case, an arbitrary number of versions can be defined that your system can know about. Why? What if you like a particular version of a package? And if you want to install a more recent version of a package? Shouldn't that be your choice? And even better yet, if you want to be running more a rolling release? This would be a way of doing it except that you don't have to upgrade ALL of your packages. You could roll back to one that works better for you....

Of course, the main problem is that someone would need to rewrite apt to use an xml file... which is probably quite a big stack - but doable.