April 2009 Archives

For one project i am currently working on i had to load and parse some binary files with ruby. The parser code was not very nice, so i came up with an idea: d = DataFormat.description do int :the_id float :value end # binary data in the stream: int(1337)float(-0.1337) data = d.read_from(stream) data.the_id #=> 1337 data.value #=> -0.1337 The cool thing about this is: The DataFormat is a description of the data format and the code to load and save a file in this data format is generated automatically. So you dont have to worry about the loading and saving code, you just have to specify the format. Some more examples: This one reads an array of values. d = DataFormat.description do uint :arr_length array(:things,:length => :arr_length) do int :thing_id float :value end end # stream: int(2)int(4363)float(0.5)int(66584)float(-1677.5) data = d.read_from(stream) data.arr_length #=> 2 data.things[0].thing_id #=> 4363 data.things[0].value #=> 0.5 data.things[1].thing_id #=> 66584 data.things[1].value #=> -1677.5 The data that is returned is an instance of the class AttributeHash. The AttributeHash just wraps an hash and overwrites method_missing to delegate all attribute accesses to the wrapped hash (so it behaves like a javascript object). But you can specify that an instance of some custom class should be created instead of an AttributeHash. Example: class ItemList attr_accessor :name, :max_size, :items end class Item attr_accessor :value end d = DataFormat.description do string :name uint :max_size array(:items,:class => Item) do string :value end end data = d.read_from(stream,:object => ItemList.new) data.class #=> ItemList data.items[0].class #=> Item data.items[0].value #=> "yes" data.items[1].class #=> Item data.items[1].value #=> "no" The requirement for the class that will be instantiated is just that is must have methods for "attrname=" calls with "attrname" specified in the data format.
The different types that are currently available to describe a data format:

Supported types
NumberSerializer: short, int, long (+unsigned: ushort etc.), float, double

uint :attribute_name, :min => 1145, :max => 67546 

StringSerializer: nullterminated(default) or fixed-length

string :attr_name, :length => 32

MagicSerializer: throws exception when the magic value (in this case 1337) is not found at the pecific position

magic :magic_number, :value => 1337

ArraySerializer: reads an array from the file. The data format of each entry can be specified.

array(:things,:length => :arr_length) do
	int :thing_id
	float :value
end

Status
Experimental version.
Todo:

• Option to specify if format is big- or little-endian
• Store errors (eg: integer out of range) in an error hash like in active record
• Optional elements (if :someattr == 0, then :otherattr is not present)
• Skip sections (if :someattr == 0, then skip x bytes)

Test syntax highlighting:
class Hash # h = {:a => 1, :b => 2} # h.extract_hash!(:a).should == {:a => 1} # h.should == {:b => 2} def extract_hash!(*keys) keys = keys.first if keys.first.kind_of? Array extracted = collect{|key,value| keys.member? key} self.reject!{|key,value| keys.member? key } extracted end end See: Max Horvath - Syntax Highlighting for your Movable Type blog

Blog created.