The first "general purpose" codemonkey plugin

[Pype.Clicker]

Hi all.

Some of you may know that, in my quest for Clicker, i've been developing a PERL-based code rewriting tool: "CodeMonkey"

The general idea is that anyone should be able to write code the way he prefers rather than the way you're expected to. It's a sort of programmable pre-processor (aimed at C-like language, though i haven't investigated whether other languages could be supported as well). CodeMonkey is just a text processing framework that will read your sources, tokenize it and do a few "managing" operations, but the 'rewriting' proper is delegated to "plugins".

You tell in your sources what plugin you want to apply, e.g.

Code: Select all

@plugin "objects.pl","synchronized.pl"

or

Code: Select all

@plugin "GTK.pl"

Then the plugin registers "keyword" on which CodeMonkey will call it back, for instance a "@synchronized" block to provide Java-like protection, etc. The most common ways to use those keywords are the "type modifiers" (e.g. having a 'public' keyword for C that add the function declaration in a .h file automatically) or "special blocks" such as @synchronized.

The reason why i'm talking about this now is that i'm just done with the first plugin that may have interest beyond my own research, e.g. processing arguments on command line. I don't know how you feel about "argc" and "argv", but honnestly, i never liked them. If you look into clicker's building tools, you'll notice weird stuff such as

Code: Select all

// options locales
clpSet clpLocal={
  "",4,"target options",{
    {"--target",setStart,"sets the default target address"},
    {"--mode",setType,"sets the module type. Valid arguments are "
     "'kernal', 'system', 'intermediate' or 'endlevel'"
    },
    {"--name",setName,"sets the module name."},
  },
  &clpExperimental
};

which is a data block to be given to a "command line preprocessor" library of my own (clp.c/clp.h) each of those setXyz names are actually functions that will be invoked once the "preprocessor" (poor name, i agree) has recognized one of the stuff (e.g. it will call setName() if it finds "--name <xxx>" on the command line).

Even with my best cpp skills, i couldn't achieve something prettier than

Code: Select all

CLPFILTER(setName) {
  CHK_ARG(1,"setting module name");
  App.name=GET_ARG(1);
  CLN_ARG(2);
}

That's small enough, but really boring to write ... and anything you find boring to write should be written by the CodeMonkey

Now, i can use e.g.

Code: Select all

@plugin "args.pl"
// some #include here

@argument int kbps=(5,"emission rate");
@argument int max=(5,"amount of packets to be sent");
@argument str ip=("127.0.0.1","target IP address");

int main(int argc, char* argv[])
{
     // local variables
     @argument.prepare;
     // now you can use kbps, max and ip ;)
}

This code will default the values of max and kbps to 5, and the value of ip to 127.0.0.1 ... I only have three types pre-defined atm (that is, decimal ints, strings and 'flags'), but anyone could define new ones provided that you edit "args.pl" and describes them in the "%types" hash:

Code: Select all

# how to process the existing types in @argument.
# new types can be added with new entries in this hash, e.g.
# - real C type for the argument (e.g. Person_t *)
# - number of arguments to retrieve on the command line (e.g. 2)
# - function or expression to create a value from the arguments
#   from strings. (e.g. newPerson(%) which will be expanded into
#    newPerson("Sylvain", "Martin") at run time.
# that would mean 
#    'person' => "struct Person_t *::2::newPerson(%)"
my %types=(
      'int'=>"int::1::atoi(%)",
      'str'=>"char*::1::%",
      'flag'=>"int::0::1",
     );

And a nice by-product is that it also automates the generation of "--help" messages

cmdemo> ./collect --help
Available Options :
Group "@args.pl auto-generated options":
* -kbps :emission rate(int)
* -max :amount of packets to be sent(int)
* -ip :target IP address(str)

Group "generic options" (--...):
* help :Display this help

If you're interested, you can download a "demo" package here.

So ... what's your general feeling about it ... Is there any things where you think you might want to have a CodeMonkey plugin or is it just overcomplication on top of an already-bloated compiler for an imperfect programming language ?

[distantvoices]

Have just scanned throu your post. That's one great tool you 've buildt there. I'm currently too much in "boah that's cooL" mode to issue anything useful besides "boah, is that cool" *rofl*

[OT] I've managed a D in maths. That's great. *gg* That's the other reason why I'm in that "Boah, that's cool" mode. Kinda lifting off, eh.[/OT]

[Candy]

The general idea is that anyone should be able to write code the way he prefers rather than the way you're expected to. It's a sort of programmable pre-processor (aimed at C-like language, though i haven't investigated whether other languages could be supported as well). CodeMonkey is just a text processing framework that will read your sources, tokenize it and do a few "managing" operations, but the 'rewriting' proper is delegated to "plugins".

<snip>

The reason why i'm talking about this now is that i'm just done with the first plugin that may have interest beyond my own research, e.g. processing arguments on command line. I don't know how you feel about "argc" and "argv", but honnestly, i never liked them. If you look into clicker's building tools, you'll notice weird stuff such as

Code: Select all

// options locales
clpSet clpLocal={
  "",4,"target options",{
    {"--target",setStart,"sets the default target address"},
    {"--mode",setType,"sets the module type. Valid arguments are "
     "'kernal', 'system', 'intermediate' or 'endlevel'"
    },
    {"--name",setName,"sets the module name."},
  },
  &clpExperimental
};

<snip>

Even with my best cpp skills, i couldn't achieve something prettier than

Code: Select all

CLPFILTER(setName) {
  CHK_ARG(1,"setting module name");
  App.name=GET_ARG(1);
  CLN_ARG(2);
}

<snip>

So ... what's your general feeling about it ... Is there any things where you think you might want to have a CodeMonkey plugin or is it just overcomplication on top of an already-bloated compiler for an imperfect programming language ?

I'm not sure but this specific example would've been rewritten differently by me:

Code: Select all

int aosmain(aos::argsmap<string, var_container> args) {
   string baudrate = args["baudrate"];

}

where argsmap would've been overridden to provide the string matching you prefer (common for args is to match a unique start as well). var_container would be a string with default operatorint, operatorbool, operator DateTime etc. to allow a plain cast to convert it to a more appropriate type including matching the locale etc.

I know, I'm offtopic.

I can see a few uses for the code monkey, but I'm not sure of anything it accomplishes that either can't be done with C preprocessor + c++ or can be done a lot nicer.

Note for the above, you could also instantiate a class to do that for you with both argc and argv as argument instead of replacing the main function - but I intended to replace it anyway, so it doesn't really matter (to me).

[Pype.Clicker]

I'm not sure but this specific example would've been rewritten differently by me:

Code: Select all

int aosmain(aos::argsmap<string, var_container> args) {
   string baudrate = args["baudrate"];
    ... 
}

where argsmap would've been overridden to provide the string matching you prefer (common for args is to match a unique start as well). var_container would be a string with default operatorint, operatorbool, operator DateTime etc. to allow a plain cast to convert it to a more appropriate type including matching the locale etc.

Hmm. yes, that's already a nice step. I have to admit i tend to underestimate the power of C++ classes when it comes to syntax sugar.
Yet, i'm a bit unsure of how you can define default values and "help messages" when one just calls "yourprog --help". Your "aosmain()" also let me things you're not targetting standard C programs, but rather AtlantisOS executables, right ?

I can see a few uses for the code monkey, but I'm not sure of anything it accomplishes that either can't be done with C preprocessor + c++ or can be done a lot nicer.

There are situations where the preprocessor cannot help, e.g. when you need to generate more than one output file (e.g. expressing class interface inline (? la Java)) or when you need to keep context information along the code rewriting.

The best illustration i could show to C++ devers would probably be the "bitfield enumeration". You know:

Code: Select all

enum inodeflags = { 
    other_exec=1,
    other_write=2,
    other_read=4,
    group_exec=8,
    group_write=16,
    ...
};

pretty boring to write... Even more if you want to have inode_other_exec, inode_other_write... to avoid namespace pollution.
For CodeMonkey, that'd be piece of cake to have

Code: Select all

bitenum inode_flags {
    other_read, other_write, other_exec,
    group_read, group_write, group_exec,
    ...
};

and have it translated properly into inode_flags_group_read = 8 ...

Note for the above, you could also instantiate a class to do that for you with both argc and argv as argument instead of replacing the main function - but I intended to replace it anyway, so it doesn't really matter (to me).

ditto here with Clicker "@program { ... }" most likely

[Candy]

I'm not sure but this specific example would've been rewritten differently by me:

Code: Select all

int aosmain(aos::argsmap<string, var_container> args) {
   string baudrate = args["baudrate"];
    ... 
}

where argsmap would've been overridden to provide the string matching you prefer (common for args is to match a unique start as well). var_container would be a string with default operatorint, operatorbool, operator DateTime etc. to allow a plain cast to convert it to a more appropriate type including matching the locale etc.

Hmm. yes, that's already a nice step. I have to admit i tend to underestimate the power of C++ classes when it comes to syntax sugar.
Yet, i'm a bit unsure of how you can define default values and "help messages" when one just calls "yourprog --help". Your "aosmain()" also let me things you're not targetting standard C programs, but rather AtlantisOS executables, right ?

I tend to syntax sugar away 90% of the complexity, so the final solution either looks great or doesn't work. Simplifying problems that way gives elegant solutions rather than butchery solutions (and designing around bugs tends to prevent them too).

There are situations where the preprocessor cannot help, e.g. when you need to generate more than one output file (e.g. expressing class interface inline (? la Java)) or when you need to keep context information along the code rewriting.

Hm.... that's a quite good point. I was considering extending the C preprocessor with vararg matching and overriding names, making it more or less a functional language. Also, it needs to support multiple lines. That makes it nigh complete for me.

The best illustration i could show to C++ devers would probably be the "bitfield enumeration". You know:

Code: Select all

enum inodeflags = { 
    other_exec=1,
    other_write=2,
    other_read=4,
    group_exec=8,
    group_write=16,
    ...
};

pretty boring to write... Even more if you want to have inode_other_exec, inode_other_write... to avoid namespace pollution.
For CodeMonkey, that'd be piece of cake to have

Code: Select all

bitenum inode_flags {
    other_read, other_write, other_exec,
    group_read, group_write, group_exec,
    ...
};

and have it translated properly into inode_flags_group_read = 8

Check. I can't beat that.