In a world gone mad..

I’ve been pretty busy with work all day. Now I got back to the IDEA Ruby plugin and after only a few minutes of thinking about what I was going to do, I said to myself “Wait a minute.. that’s BS..”. There’s a much simpler solution..

Alright, here’s the context: Yesterday I spent a few hours trying to use a ‘search-replace’ approach for turning the JavaScript IDEA example plugin into a Ruby plugin. Didn’t work.. :) And after a few minutes playing around with JFlex.. well, it really was enough JFlexing for a lifetime.. I then started implementing the Lexer interface offered by IDEA again, keeping in mind that at some point I’d have JRuby doing the ‘tokenizing’.

But while doing that, I ‘stumbled’ across a simpler idea. How about a concept called ‘RecognizedToken’, offering a simple API like this:

interface RecognizedToken
{
    boolean isFoundIn( CharSequence, StartOffset, EndOffset );
    int getTokenStart();
    int getTokenEnd();
    IElementType getTokenType();
}

Then it should be possible to implement a very simple Lexer checking all registered tokens on every ‘advance’ call. This Lexer could read a simple config file consisting of lines such as:

LINE_COMMENT => "#.*"
DOUBLE_QUOTED_STRING => "\"(?:[^\"]|\\\")*\""
(...)

Of course, the next problem is the complexity of the regular expressions. And performance could be an issue, if IDEA doesn’t properly cache the Lexer output. But judging from the ‘start’ method signatures, my guess is they will cache everything..

Anyway, the Lexer now becomes a simple “10″ liner. Well, apart from the 100 LOC used to implement the generic API..

private final void updateTokenType( final int aStartOffset )
{
    myTokenStart = aStartOffset;
    myTokenEnd = myEndOffset;
    myTokenType = null;

    final RecognizedToken recognizedToken = myTokenFinder.findClosest
        ( myCharSequence, aStartOffset, myEndOffset );
    if ( recognizedToken == null ) return;

    myTokenType = recognizedToken.getTokenType();
    myTokenStart = recognizedToken.getTokenStart();
    myTokenEnd = recognizedToken.getTokenEnd();
}

The basic Lexer is working now. I’ll add configuration file support and then try some Ruby coding in IDEA with this plugin’s syntax highlighting enabled.

tfdj

Filed by The.French.DJ at 7:12 pm under Development,Java,Ruby
No Comments