Just stumbled upon a – slightly weird – mentioning of JRuby on Lambda. It’s weird because JRuby has been around for a long time..
Anyway, I’m just thinking, maybe this could be a nice answer to my recent ‘performance’ problems mentioned here. Using the Java NIO memory-mapped files stuff in (J)Ruby..
Overall a very cool project.. imho..
tfdj
Filed by The.French.DJ at 3:39 pm under Development, Java, Ruby
No Comments
My favourite Java IDE is JetBrain’s Intelli/J IDEA. But these days I wonder about it’s future. Not because of Eclipse. Eclipse – for me – is a different approach to an IDE and I simply don’t like it. But that’s not the point. The point is that IDEA is getting more and more bloated. Probably not only because of features, but also because of the included JDK/JRE. Not sure on this..
Anyway, I simply don’t need about 50% of the functionality. IDEA has become a monster like Word. And I think this sucks.
I want an IDEA Light. Possibly without an embedded JDK/JRE. As a compiled executable or something like that. And less than 10 MB total size. The core editor and a Plugin API. Obviously with some serious limitations compared to the bloatware IDEA. No UI designer, no J2ME support, no J2EE support, nothing like that. Java editing with Java project-file management and the strong and beloved refactoring support. That’s probably all we need in a neat package.
Kind of like a tight JEdit with the known cool Intelli/J UI.
Well.. Just a thought..
tfdj
Filed by The.French.DJ at 7:52 pm under Development, Java, Rant
No Comments
For the project mentioned in my previous post, I needed a way to ‘clip’ polygons. Polygons had to be intersected with a ‘bounding box/shape’ and they had to be triangulated.
I was faced with the decision to either implement one of the known algorithms, find a Ruby polygon clipper, use an external implementation, write a Ruby extension providing access to one of the available c/c++ implementations.
Performance is/was an issue. Geo data polygons can contain a large number of coordinates. And many polygons have to be processed.
I immediately dropped the idea of writing a pure Ruby implementation. Mainly because I’m new to Ruby and I don’t dig math at all.. :/
I found no Ruby implementation after a few hours of searching the web.
Then I started calling external implementations of ‘hgrd’, ‘gpc’ and some other polygon clippers via a primitive ‘exec’ call from Ruby. Obviously the performance was lousy. But I was able to deliver some first results quickly.
I also realized that there a huge differences in clipping quality. And I finally settled for the ‘GPC’ from Alan Murta: http://www.cs.man.ac.uk/~toby/alan/software/
After a few days I had to improve performance. So I started looking a ‘extending Ruby’.
I needed a few hours and a look at swig to finally ‘grok’ the way extending Ruby works. My first approaches – before swig – failed badly. I tried to map all ‘native’ objects’ to Ruby somehow, providing all functions somehow.. a total mess..
Looking at swig I was shocked by the ugly code. But at least I was able to derive some ideas. Interestingly enough, the swig code didn’t really work. It didn’t ‘understand’ the ‘pointers’ as ‘arrays’ used by the GPC. Or to be more precise: I didn’t understand how to tell swig what to do..
Anyway, after playing around with this code a few hours it finally clicked. It was then a matter of a few more hours to finish the extension using very simple – imho – code.
Here’s the code in case you’re interested:
http://www.berlinfactor.com/blogging/files/gpc_ruby.c
I should probably send this to Alan Murta. But for now it’s to ‘raw’. No index/error checking at all..
tfdj
Filed by The.French.DJ at 12:59 pm under Development, Ruby
2 Comments
I was lucky enough to be able to choose Ruby for implementing a small server for ‘geo data’. Basically a service not unlike Google Maps. But much more simple and with vector output. The whole thing is used for providing data that can be used on a J2ME client.
Anyway, because of some weird problems inside the company, I had to read a 2 GB big file containing the geo data in a stupid textual format.
I started with a trivial ‘each_line’ approach. A ‘data entry’ in the file consisted of about 10 lines. The first 9 lines contained some attributes. The last line contained the coordinates of the represented geographic feature. So within the each_line block, i collected the data and attached a resulting ‘GeoObject’ instance to an output array whenever a data entry has been read completely.
This thing took about 4 hours to process the 2 GB.. wtf!? Alright, I didn’t think much when writing the code. And I only worked with Ruby for a few days. But some things were obvious to me: For example extracting all the ‘values’ out of a ‘line’ by extracting a string and calling ‘to_i’ is pretty inefficient.
So I started implementing a few straight forward optimizations: Instead of extracting substrings from the line I directly add the ‘bytes’ to determine the integer value. And instead of ‘each_line’ I read 16 MB chunks of data and worked with offset/index pairs on these chunks.
This improved performance by more than 50%. But still close to 2 hours.
Funny me, I fired up Intelli/J IDEA. This took about 2 minutes.. (I hate how bloated IDEA is by now.. I’d love to see an IDEA Light!) And I started hacking away a Java solution. Using IDEA this took me less than 30 minutes for this problem. I used the NIO features. With ‘getChannel’ and some ‘map’ call to do memory mapped IO my first version of this app took about 5 minutes to process 2 GB.
How’s that?
I can’t explain all of this huge performance difference.. A part might be the memory mapped IO. But look at this Java API. Scanning the ByteBuffer using these ‘get’ calls.. I assume the Java VM is a lot more powerful than what Ruby’s ‘foundation’ offers.
Anyway.. just a quick post on this topic. If I find the time I’ll post the code. Unfortunately I have to change some things to protect the innocent..
I’ll post another note on extending Ruby soon. I needed access to a polygon clipping library.. More on that soon..
tfdj
Filed by The.French.DJ at 12:58 pm under Development, Ruby
No Comments
alright.. a little gem.. you know about bookmark keywords, right? so you got about 1 million bookmarks (why?) and 10 of them are somewhat important. with firefox (and other browsers :) you can give those important bookmarks a keyword. then you enter this keyword in the address bar of the browser and the associated bookmark is loaded. you could use ‘gn’ for ‘news.google.com’ for example.
well, dopus let’s you do this on your file system. there are ‘favorites’ (bookmarks) in dopus. and you can define aliases. then you enter ‘/’ in the lister address bar and you navigate to ‘/some/really/long/path/somewhere/else’.
just a little feature.. but sooo good to have.. i use this for example to alias certain locations on my samba server’s file system. like the main sections of my web-sites, or my cruisecontrol home, and so on..
tfdj
Filed by The.French.DJ at 12:57 pm under Uncategorized
No Comments
if you’re still using the windows explorer for file management or if you’re still stuck with one of the old school file management solutions (total/speed commander and the like) you simply have to check out directory opus: bringing the file management solution from the old amiga days to the pc and taking things a step further every day.
dopus is not limited to a single window with two lister panes. dopus replaces the windows explorer with new improved explorer windows.
i think dopus compares to all other file management solutions like intelli/j idea compares to eclipse. it’s not a generic file management framework where you can plugin tools and stuff in generic ways. but it is an integrated solution that has defined ways of extending it by plugging in tools and scripts.
there are pros and cons about this. above everything else i’m missing a nice lister/vfs api. one of the strengths the other file managers have: you can write a WebDAV vfs plugin if you want. but then again, i’m not missing it that much.. because dopus is just sooo useful with its built in feature set.
i’ll blog about some of these features from time to time, to share my excitement with you.. :)
tfdj
Filed by The.French.DJ at 12:55 pm under Uncategorized
No Comments
Following this information
http://www.brembs.net/litestep/win2k.html
it is quite easy to replace the windows explorer.exe with for example a kiosk.bat from which you could start Firefox or whatever tools you want to be running in ‘kiosk mode’.
Some additional reg hacks make sure the taskmanager is disabled. Setting some file permissions makes sure the file system can’t be tampered with.
Obviously this is still a very insecure system. But it fits my needs for setting up a laptop in kiosk mode for a small espresso bar around the corner.. The bar, btw, is highly recommended: www.trescabezas.de If you should ever be in the east part of Berlin, drop by. I’ll probably be around.. :)
tfdj
Filed by The.French.DJ at 12:43 pm under Uncategorized
No Comments