Part I – A long blown out guide to interpretting ones and zeros through an account of my own experiences involving out-dated software.

 It’s been a while. If fact it’s been so long I had to dig through my emails just to find the wordpress password. My last published post was sometime in April, 2009 and I haven’t logged back in much since then but, as I roamed about the admin section I found an unfinished post on a central locking circuit I made for a friends car. I finished that project late last year so I will have to dust it off in the near future and publish it.

Anyway, apart from a few projects on the go the last couple of my weeks have been dedicated to reverse engineering an old MS DOS schematic capture file format. I am not one to look for obscure file formats to break apart and so there is a motive behind this relatively tedious project.

On the motivation behind this relatively tedious project

Like most other beings I spend my days doing work, though unlike the larger portion of other beings part of my work requires electrical system schematics to be drawn. Well, perhaps there is a fair sized portion of employed people using schematic capture software but what makes my experience unique is that its done in an MS DOS program called Protel Schematic V3.31. For those who are thinking, so what?! Its 2011, that’s what! And, my knowledge of MS DOS history is pretty pour but by the time I could use a computer Microsoft had moved onto Windows 98. Though, with that said I understand that there are many companies out there using antiquated technology and, perhaps it’s for the simple reason that they are tools that work and are easy to use.

If the system ain’t broke don’t try to fix it and its true, why churn out thousands of dollars to replace a tool that all your staff are trained on and can use efficiently? The answer in this case is that Windows 7 won’t run Protel Schematic V3.31 with out running it through an emulator of some kind.

Since its debut in the 80s Protel, an Australian company, has been bought out by Altium who promptly put a stupendous price tag on their line of EDA software currently called “Altium Designer 10″. Even though it does open old Protel files it would be of no use to you because you have just sold your liver and three of your kidneys to get a license that lasts only a year.

There are plenty of cheap options out there, though trying to find one without all the bells and whistles is hard, since we do system design and generally don’t get down to circuit level and PCB design and testing. After some searching I found KiCad, its not too ugly to look at and, its free which means as I pryed apart the layers of Protel schematic files I had a format to convert it too. The beauty of KiCads’ schematic file format is that its ridiculously simple to interpret and since it’s in plain text, if you got your geek on and neglected life for a bit, you could write your own schematic by hand.

This has been my first real experience with the process of reverse engineering and so this article is going to serve the purpose of passing on the things I have learned, as well as break down and document an otherwise undocumented Protel Schematic V3.31 file format.

On reverse engineering and using your head

Two factors in this process of reverse engineering have made my life easy, for one I have the software that generates the file format, secondly the format that I am converting to is well documented. Having the software to generate the file format means that you can control the data stored in the file, take one of our large system schematics for example; it contains a couple of hundred lines, junctions, text labels, a document box and components, that’s a lot of bytes to process.

One tool that will be invaluable is a decent hex editor, on windows I use 010 Editor and on the mac I use hex fiend (pretty basic but it works).

I began by opening up the schematic file in the hex editor which shows gibberish nonsense when looking at it in ASCII and plain text. Having a split screen of the file, one side translated in ASCII and the other in hex helps pick out patterns.

ASCII will encode what it can into characters and in some parts of the file  words are formed which gives clues at to what is contained in that section. It allowed me to pick up where labels and components were located in the file.

HEX is a base16 numbering system using 0-9 and A-F, so instead of trying to find patterns in large amounts of binary it it stores groups of 4 bits as a single value.

With that initial formatting done large blocks of data stand out and it is easy divide the file up into its patterns and start to get a grasp on how the file is structured.

As I mentioned before, having the software that creates the file format allows you to have control over the data that is stored and so, by saving a blank schematic you can see what data is related to the document setup, formatting and what is used to define the other objects on the page.

On Ruby and processing files bit by bit

Using Ruby to write the file converter made things simple since it has some useful tools for reading data byte by byte.

Initially with when loading the file into Ruby we set it to binary mode, this means that it won’t be automatically encoded or use any ASCII newline commands that may be in the file and also reads the file as ASCII-8BIT. This is done using the File subclass of the IO class.

schematic = File.new(ARGV[],"r")
schematic.binmode

The IO class also provides us with another function called seek and this allows us to jump to certain locations within the file, as well as remember its current location.

In the example given below, I have found out how many bytes in size the header information for the schematic file is (16 bytes), so using the seek function I move past it and then read the next 16 bytes of data.

#our position within schematic is 0
schematic.pos  #=> 0
header = 0x10
schematic.seek(header, SEEK_CURR)
schematic.pos  #=> 16
schematic.read(16)

The next part to this article is going to be a break down of the file format and other technical stuff such as that…