Not in fact any relation to the famous large Greek meal of the same name.

Wednesday 6 October 2010

Quirking a Mac Pro into AHCI mode

The Apple Mac Pro (2006 model) has an Intel ESB2 south-bridge, which includes the SATA controller. Under MacOS, this SATA controller is driven in AHCI mode, which is a great improvement on the “traditional IDE” mode — for one thing, it enables “NCQ” SCSI-like command queueing. But the pseudo-BIOS that boots non-MacOS operating systems on Intel Macs, forces the chip into traditional-IDE mode. This is certainly a good idea for wide compatibility, but for an OS which has AHCI drivers — such as Linux — it’d be good to force it back again.

Instructions do exist for forcing it back again using a hacked Grub bootloader, but for one thing that’s a hack, and for another I don’t use Grub, I use Lilo. The “right” way (well, other than convincing Apple to put an AHCI mode into their pseudo-BIOS) is to use Linux’s “PCI quirk” infrastructure. This patch (against 2.6.35.6, but similar should apply almost anywhere) gives me toasty AHCI goodness under Linux:

--- drivers/pci/quirks.c~ 2010-09-27 01:19:16.000000000 +0100
+++ drivers/pci/quirks.c 2010-10-06 20:29:04.000000000 +0100
@@ -1044,6 +1044,15 @@ DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDO
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_HUDSON2_SATA_IDE, quirk_amd_ide_mode);
 DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_HUDSON2_SATA_IDE, quirk_amd_ide_mode);
 
+static void __devinit quirk_intel_esb2_ahci(struct pci_dev *pdev)
+{
+    pci_write_config_byte(pdev, 0x90, 0x40);
+    pdev->class = PCI_CLASS_STORAGE_SATA_AHCI;
+    dev_info(&pdev->dev, "Intel ESB2 AHCI enabled\n");
+}
+
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2680, quirk_intel_esb2_ahci);
+
 /*
  * Serverworks CSB5 IDE does not fully support native mode
  */
And here it is in my dmesg:
pci 0000:00:1f.2: Intel ESB2 AHCI enabled
ahci 0000:00:1f.2: flags: 64bit ncq pm led slum part 
ata1.00: 488397168 sectors, multi 16: LBA48 NCQ (depth 31/32)

All six SATA ports (four for the drive carriers, two by the optical drives) are seen by the ahci driver (which I have built-in to the kernel); loading the ata_piix driver as a module also enables the PATA port by the optical drives. (It's probably not a good idea to compile both ahci and ata_piix into the kernel, in case they fight over the ESB2 controller.)

Job’s a good ’un.

Monday 31 May 2010

Portability, Or Cut-And-Paste?

Suppose you’ve got some functionality with two completely different implementations on different platforms:

// Event.h v1
class Event // Win32 waitable event
{

  void *m_hEvent;
public:
  Event() : m_hEvent(::CreateEvent()) {}

  void Wake();
  ...
};
versus
// Event.h v2
class Event // Posix waitable event
{
  int m_fd[2];
public:
  Event() { ::pipe(m_fd); } // or maybe eventfd on Linux
  void Wake();
  ...
};

And, by the way, if you’ve got a class where only some of the implementation is different on different platforms, you’ve first got an opportunity for refactoring to extract that part of the implementation as a self-contained class. Portability and maintainability are aided if each of your classes is either wholly portable, or wholly platform-specific. But as it is, you’ve got a more immediate problem, which is that there’s two classes both called Event, and any translation unit that includes both headers isn’t going to compile. Here’s one solution:

// Event.h v3
#ifdef WIN32
class Event
{
  ...
};
#else
class Event
{
  ...
};
#endif
This, plus its close relative
// Event.h v4
#ifdef WIN32
#include "win32/Event.h"
#else
#include "posix/Event.h"
#endif
could well be the most commonly-encountered solutions to this problem in real-world code. But they’re not without problems. One problem is with code analysis tools, including such things as production of automated documentation with Doxygen. Really you want such tools to analyse your whole codebase, and if they do they’ll get hopelessly confused with the two classes both called Event. (You can set up preprocessor defines in Doxygen so that it only sees one Event class -- but that only means the other doesn’t get documented, or that you have two entire sets of Doxygen output for the two platforms, neither of which sounds desirable.)

To solve this problem, you’ve going to have to give the classes different names:

// Event.h v5
class Win32Event
{
  ...
};

class PosixEvent
{
  ...
};

No, no, no, this is C++, there’s a built-in language facility especially for namespaces, you don’t have to reinvent it in the class names themselves:

// Event.h v6
namespace win32 {
class Event
{
  ...
};
} // namespace win32

namespace posix {
class Event
{
  ...
};
} // namespace posix

(Why the comments on the “}” end-of-namespace lines? Because the error messages you get from C++ when you accidentally miss out an end-of-namespace brace are often insane and impenetrable, especially if you do it in a header file, and it’s convenient to be able to grep namespace *.h *.cpp and check whether braces appear in matching pairs in the grep output.)

Notice that both declarations are parsed on every compilation on either platform. This might require you to move some method definitions (those calling such unportable APIs as ::CreateEvent and ::pipe) into a cpp file. But even compiling the other platform’s declarations helps ensure that changes on one platform won’t break the other platform without somebody noticing. And a Doxygen run, or other automated code analysis, will at least get to see both sets of declarations even if not both sets of definitions.

While going down the route of helping platform developers avoid breaking other platforms, it’s been conspicuous so far that there’s nothing in Event.h making sure that the Win32 and Posix implementations keep the same API. Without such enforced consistency, there’s a risk of client code unwittingly using non-portable APIs. The way to enforce the consistency, of course, is to derive from an abstract base class containing the API:

// Event.h v7
class EventAPI {
public:
  virtual ~EventAPI() {}
  virtual void Wake() = 0;
  ...
};

namespace win32 {
class Event: public EventAPI
{
  ...
};
} // namespace win32

namespace posix {
class Event: public EventAPI
{
  ...
};
} // namespace posix

This is, after all, exactly the sort of thing you’d do if you had two different sorts of Event that were both used in the same build of the program. Really the fact that no one individual binary will contain instances of both classes, doesn’t mean that the quest for well-factored design should be thrown out of the window and replaced with cut-and-paste.

Now, though, all the method calls have become virtual function calls. For most of any given codebase, that won’t matter, but there’ll always be hot paths and/or embedded systems where it does, and indeed an event class might quite plausibly be on such a critical path. And after all, compiling the client code itself provides a, potentially large, body of checks that the API has remained consistent over time. It’s reasonable to adopt the position that using design techniques to discourage unthinkingly changing core APIs in a non-portable way isn’t actually necessary, so long as the compilation-failure results of such a change are always speedily available from all target platforms, such as from an automated-build or continuous-integration system.

If, conversely, your class is only used in situations where nobody’s really counting individual machine cycles, you could hide the implementations altogether, at the cost of ruling out stack objects and member variables of type Event and requiring a heap (new) allocation every time one is created:

// Event.h v8
class Event {
public:
  virtual ~Event() {}
  virtual void Wake() = 0;
  ...
};

std::auto_ptr<Event> CreateEvent();

// Event.cpp
std::auto_ptr<Event> CreateEvent()
{
  ... return whichever derived class of Event is appropriate ...
}

But for now let’s assume you can’t really conceal the two platform-specific declarations, and go back to Event.h version 6 or version 7. Let’s look at how client code gets to pick which of the two implementations it uses. For a start, not like this:

class NominallyPortableDomainAbstraction
{
#ifdef WIN32
  win32::Event m_event;
#else
  posix::Event m_event;
#endif
  ...
  m_event.Wake();
  ...
};

Flouting our design rule that each class is either wholly portable or wholly platform-specific, this sort of thing couples portability decisions into all client code, which is very undesirable. This is much better:

// Event.h v9
... as before in v6 or v7

#ifdef WIN32
typedef win32::Event Event;
#else
typedef posix::Event Event;
#endif

// Client code
class NominallyPortableDomainAbstraction
{
  Event m_event;
  ...
  m_event.Wake();
  ...
};

It’s still not great, though: if there are lots of classes involved, lots of code ends up inside the #ifdefs. How can we minimise the amount of #ifdef’d code? Well, here’s one way:

// Event.h v10
... as before in v6 or v7

#ifdef WIN32
using namespace win32:
#else
using namespace posix;
#endif

But this, of course, introduces everything from those namespaces into the surrounding namespace. Not only is that namespace pollution, it’s potentially misleading for client code, as there may be Win32-specific classes or functions -- ones with no Posix equivalent, for use in Win32 situations only -- which are now accessible in portable code without the telltale win32:: prefix. (And, of course, vice versa for Posix-specific ones.) Really we want to explicitly enumerate which classes and APIs are intended to be portable. So we can do this:

// Event.h v11
... as before in v6 or v7

#ifdef WIN32
namespace platform = win32;
#else
namespace platform = posix;
#endif

using platform::Event;
using platform::PollThread;
...

The intention is that the using declarations list precisely those facilities which are available on all platforms, but with differing implementations on each. Non-portable classes or functions can stay in namespace win32 (or namespace posix) so that any use of them in client code immediately flags that code up as itself non-portable.

There is one problem with this neat solution, though: it doesn’t actually compile. Or rather, once you do the same thing in several different header files, you’ll find that multiple namespace X = Y; statements for the same namespace X aren’t allowed, even if Y is the same each time. So, sadly, you have to come up with a different name for “platform” each time (or centralise the using in one header file, causing potentially-undesirable widespread dependency on that file):

// Event.h v12
... as before in v6 or v7

#ifdef WIN32
namespace eventimpl = win32;
#else
namespace eventimpl = posix;
#endif

using eventimpl::Event;
using eventimpl::PollThread;
...

There’s only one remaining wrinkle, which is that you can’t forward-declare a class-name if that name only exists due to using. So if there are header files that traffic in Event* or Event& and could otherwise be satisfied with a forward declaration class Event; and avoid including Event.h, then you can’t use Event.h version 12. (Version 9 is also out, as you can’t forward-declare typedefs either.) The best you can do is probably this:

// Event.h v13
... as before in v6 or v7

#ifdef WIN32
namespace eventimpl = win32;
#else
namespace eventimpl = posix;
#endif

class Event: public eventimpl::Event {};
class PollThread: public eventimpl::PollThread {};
...

although naturally that only works as-written if the eventimpl base classes don’t have constructors other than the default constructor; if they did, you’d have to write forwarding constructors in each derived class, making the code a lot less neat.

Wednesday 26 May 2010

Think Same: Mapping An Apple Keyboard Like It’s A Normal One

The aluminium “laptop-style” Apple keyboards are really nice: quick and easy to type on. I’ve got one on my main Linux machine. But they’re mapped oddly (or at least the UK one is): backtick is next to Z, and backslash next to Enter. As I’m usually watching the screen and not the keyboard while typing, the fact that these think-different mappings are borne out by the printing on the keycaps doesn’t help.

So I needed to swap them back again. And, X being the stovepipe it is, I needed to do it twice, once for inside X (including Gnome and KDE) and once for the console.

X was actually easier: I added this to ~/.xinitrc:

# Apple silver keyboard has these keycodes swapped
xmodmap -e "keycode 94 = grave notsign grave notsign bar bar"
xmodmap -e "keycode 49 = backslash bar backslash bar bar brokenbar"

For console use there doesn’t seem to be a similar way of modifying just certain keycodes, so you end up needing to make a whole new keymap:

dumpkeys > keymap.txt
(edit keymap.txt)
loadkeys < keymap.txt

Here’s the diff I had to apply to the standard UK map to get the Apple keyboard working the way my fingers expect:

--- std_keymap  2010-03-12 13:49:34.000000000 +0000
+++ silverapple.keymap  2010-03-12 13:50:39.000000000 +0000
@@ -93,9 +93,9 @@
        shift   control keycode  40 = nul             
        alt     keycode  40 = Meta_apostrophe 
        shift   alt     keycode  40 = Meta_at         
-keycode  41 = grave            notsign          bar              nul             
-       alt     keycode  41 = Meta_grave      
-       control alt     keycode  41 = Meta_nul        
+keycode  86 = grave            notsign          bar              nul             
+       alt     keycode  86 = Meta_grave      
+       control alt     keycode  86 = Meta_nul        
 keycode  42 = Shift           
 keycode  43 = numbersign       asciitilde      
        control keycode  43 = Control_backslash
@@ -201,10 +201,10 @@
        control alt     keycode  83 = Boot            
 keycode  84 = Last_Console    
 keycode  85 =
-keycode  86 = backslash        bar              bar              Control_backslash
-       alt     keycode  86 = Meta_backslash  
-       shift   alt     keycode  86 = Meta_bar        
-       control alt     keycode  86 = Meta_Control_backslash
+keycode  41 = backslash        bar              bar              Control_backslash
+       alt     keycode  41 = Meta_backslash  
+       shift   alt     keycode  41 = Meta_bar        
+       control alt     keycode  41 = Meta_Control_backslash
 keycode  87 = F11              F23              Console_23       F35             
        alt     keycode  87 = Console_11      
        control alt     keycode  87 = Console_11

And to get this loaded on every boot, I stuck it in /etc/rc.local:

# Apple keyboards start up in "F-keys are magic" mode; this puts them in
# "F-keys are F-keys" mode
echo 2 > /sys/module/hid_apple/parameters/fnmode

# Also they have the backslash and backtick keys swapped
loadkeys < /etc/silverapple.keymap

Oh, right, yes, the function keys: they think different too, or at least they default to think-different mode on power-up. To get them thinking same, you need to have the Linux hid_apple module loaded, and to set its “fnmode” parameter as above. (Other systems may use other ways of setting module parameters.)

If you’ve found this and want to check that your fingers are mapped the same way mine are, this is the list of keys that, with the setup above, do not generate the symbol printed on the keycap:

labelledgenerates
§` (backtick)
± (shift-§)¬ (notsign)
@ (shift-2)"
" (shift-')@
\#
| (shift-\)~
` (backtick)\
~ (shift-`)|

Overall you lose the ability to type “§” and “±”, and gain “¬” (meh) and, rather surprisingly, the hash character “#”, which isn’t marked anywhere on the Apple keyboard.

Sunday 31 January 2010

How Facebook Wasted An Hour And A Half Of My Life

Now it should be admitted up-front that in fact, taken as a whole, Facebook has in fact wasted far more than an hour and a half of my life. But all the other hours and halves, I at least felt it was my choice to waste them. This time, however, it was definitely Facebook’s choice instead — or Apple’s.

The problem started when Facebook for Iphone refused to do anything, showing only the message “This version of the Facebook application is no longer supported. Please upgrade to version 3.0.”

Despite having always thought that the whole point of web applications, or websites in general, is that you design them carefully-enough to start with that you can support old URLs or clients in the field indefinitely — you can’t, after all, reach out and rewrite everyone in the world’s bookmark files if you want to change a URL — I consoled myself that at least updates to Facebook for Iphone are free, so went off to the Iphone App Store to download the newer version. But that failed too: apparently version 3.0 of Facebook for Iphone required at least Iphone OS 3.0, and mine was still on 2.2.

Upgrading the OS on a jailbroken Iphone can be stressful, but I thought that, as Facebook was pretty much the only Iphone application I actually used, even the worse-case scenario of bricking the phone and going back to the Nokia 8890 wouldn’t lose me that much. It turns out that the procedure for upgrading a jailbroken Iphone to OS 3.1.2 is as follows:

  • Find out how. I’m not even including the finding in the hour and a half.
  • Reboot, because it only works from MacOS.
  • Download Pwnagetool, BL-3.9, and the IPSW. (I always think that the file icon for IPSW files ought to be a blue scarf, but it isn’t.)
  • Try and download BL-4.6, but the link doesn’t work.
  • Go and fetch BL-4.6 on a USB stick from the Windows laptop you last did an Iphone upgrade from. (This is actually an improvement over the previous process, which only lets you know you’ll need the BL files halfway through the upgrade itself.)
  • Run Pwnagetool, which immediately stops because you need at least Itunes 8 for the “restore from specific IPSW” feature, and you only have Itunes 7.
  • In Itunes, run “Check for updates”, which runs the system Software Updater program, which offers loads of nonsense. Untick everything except Itunes, and download and install it.
  • Run Pwnagetool again, which now does everything it needs to and creates the custom IPSW.
  • Put the Iphone into recovery mode.
  • Run Itunes, which refuses to start because it needs Quicktime 7.5.5 or later.
  • Go and find Software Updater (it’s in System Preferences) and run it again. Untick everything except Quicktime, and download and install it.
  • Reboot, because the Quicktime installer demands it.
  • Put the Iphone into recovery mode again because it’s fallen out of it during the reboot.
  • Re-run Itunes, and have it tell you “Itunes could not contact the Iphone software update server because you are not connected to the internet.”
  • Click on “More information” about this “no internet” error, and watch it open a web page in Firefox to tell you about it.
  • Start to wonder whether Apple are just deliberately messing with your head in order to scold you for jailbreaking their phone.
  • Unplug the network cable from the Mac and plug it directly into the router, bypassing the HTTP proxy.
  • Reconfigure both MacOS and Firefox not to use the proxy.
  • Quit and re-run Itunes, because there’s no obvious way of telling it to rescan for devices.
  • Actually install the upgrade, which takes ages to “prepare”, much of which time you spend eyeing the old Nokia 8890 which never caused you any of these problems.
  • Once the Iphone restarts, observe with relief the little magnifying-glass icon that 2.2 didn’t have, thus providing evidence that the upgrade actually did something. Also notice that it’s rearranged all your icons; arrange them back the way you like.
  • Install the Facebook application from the Iphone App Store.
  • Open “Contacts” and realise that in fact the upgrade has wiped all your user data.
  • Notice with relief that Itunes is now asking you whether to restore the Iphone from a backup.
  • Notice with alarm that the backup in question is from 2008.
  • Restore from the backup anyway because it’s got to be better than nothing.
  • Rearrange all your icons back the way you like them again.
  • Reinstall the Facebook application from the Iphone App Store, because the restore restored the old version.
  • Unplug the network cable again and plug the Mac in back behind the firewall where it belongs.
  • Reboot back into Linux.
  • Start wondering what important phone numbers you’ve obtained since 2008.

About Me

Cambridge, United Kingdom
Waits for audience applause ... not a sossinge.
CC0 To the extent possible under law, the author of this work has waived all copyright and related or neighboring rights to this work.