Interesting and potentially useful insights, thank you. Let me see if I can apply/reply to them in turn, and see what the response is.
cxzuk wrote:1) Understanding the Continuum,
This was a huge win for me personally, on understanding what I was actually doing, and what i was designing. It was a bit like a separation of concerns.
Models (Memory Model, Concurrency) <--> Expression Construction Tools (syntax) <--> Development Environment (IDE) <--> Configuration (Deployment) <--> Models
I have given a lot of thought to all of these issues, but I hadn't really tried to codify them this way. That's something I ought to try, thank you.
cxzuk wrote:TL;DR; Get a good understanding of what bits you're designing. I kept too much of a mishmash of ideas without understanding how they relate.
This is very relevant. I have spent too much time thinking about things without nailing down the details and getting them down in writing.
cxzuk wrote:2) Ditch BNF (Your milage may vary, my language is object-oriented and has statements as well as expressions.)
I have come to the conclusion that BNF is a waste of time. My reasoning is that what it is trying to describe is functional composition, but it omits details and makes this more confusing that it really needs to me. I imagine 90% of people will want to right a Recursive-Descent parser anyway so map your syntax design to what you'll actually be writing. I decided to go with XML/XSLT to describe my syntax and transform that into my code required to parse.
I required additional information, most notably naming each matched statement. If that statement was optional or not, and control-flow constructs (one-or-more, one-of which map to symbols used in BNF).
TL;DR; BNF simplifies too much (especially for statement based languages), and control-flow isn't clearly handled in BNF.
I have to disagree with you here, but I suspect that it also reflects a misunderstanding of BNF itself. Backus-Naur Form is for defining lexemes and syntax, and nothing more; It serves mainly to formalize the design of the lexical analyzer and the parser. It sounds as if you were trying to apply it to the semantics, which is simply a bad idea all around. Abstractly at least, the parser shouldn't even know things like element matching, or which construct is a conditional and which aren't. While those all get knotted up together in a most direct-emit recursive-descent parsers,
In any case, I was only using EBNF to define the lexemes, as parsing an s-expression language is pretty much trivial, even where read macros are used in the program. Most of the things that would have a special syntax for in conventional languages, such as assignment, variable definition, arithmetic expressions, conditional constructs, and so forth are all done with s-expressions instead, and if a syntax is really needed, it can be defined programmatically with read macros. For example, rather than the infix expression
a Lisp would use
where 'define' and '*' are standard forms (or in some cases, library procedures). The general syntax of '(<function> <arg-1> <arg-2> ... <arg-n>)' is pretty much the basis of all s-expression languages, Lisp or not (most s-expr languages are List processing languages, but some, like TRAC or my own language, Thelema, are not).
This is especially true in this case, since the core language has no literal forms at all; numeric literals, string literals, character literals, and anything else I choose to have a literal syntax for will be defined in a combination of macros and specializers.
cxzuk wrote:3) Get to code already
I totally regret spending so much time in the design phase and working on paper. I also regret trying to code in C++ from the get go. I ditched both of these things for off the shelf tools and tools that were very forgiving, Current compiler is written in Javascript, XML and XSLT. - I am much more interested in the flexibility to try new things quickly than anything right now. And you can start talking about what it DOES instead of what it will do. You really can't tell from paper alone how well something goes.
TL;DR; Pick tools that suit you're design needs and get coding already.
Getting to coding already is good advice, bit not always easy for me. I tend to get nervous, and put things off in hopes of finding a better solution. As for my toolchain, I intend to write those myself; am currently bootstrapping my assembler, Assiah, in R6RS Scheme, and will do the same for the bootstrap version of Thelema, but as soon as I get a working version I intend to re-write it to be self-hosting.
