[dartoscoder]

Not that I actually plan to make one in the near future it is still a topic that intrests me.
I know that, to some extent, programming languages evolve like normal(for lack of a better word) languages do. Ex: C evolved into C++.

So how would you go about making a whole new one like M$ did with F# or how the creator of C did it.

Just wondering threads

[blackcompe]

You have to build a compiler that can generate machine code from your specific high-level instructions, or you have to build an interpreter which will execute your instructions for you. I have a tutorial on the first phase of compiler design if you care to look. Stroustrup, Ritchie, Gosling: they all built compilers.

[jon.kiparsky]

Before you build the compiler, though, you have to build the language. A compiler is simply a machine for turning valid statements of language L into executable code for machine M. The language itself is what defines the work of that compiler.

If you want to define a language, I frankly don't know where you'd start. I do know, however, that you'd want to write compilers for a few existing languages first. You want a language that's easy to parse and easy to compile, and writing a compiler gives you an idea of what's easy and what's not. It also requires that you study some theory that you probably don't have yet - get the Dragon book, or if you're into Java get Bill Campbell and Swami Iyer's forthcoming book on compiling Java.

Once you've learned how compilers work, you might want to think about what it is you want your language to do. What's the big idea?
In order to get this, you should probably learn a lot of languages. Certainly I wouldn't even think about making a new language without having studied a long list of existing ones, including all of the obvious ones that we talk about all the time here, and also less obvious stuff.

Those steps will probably occupy you for a little while - you'll likely have a better idea of how to proceed once you've finished them.

[Shane Hudson]

I have been looking into compiler and language design on a very basic level, while I cannot answer your question in one reply... for you and anyone else interested (it is one of the more advanced subjects) check out this Stack Overflow thread.

[blackcompe]

Jon brought up a good point about defining what the language actually does before implementing it. Sebesta is a good starting point for the design of programming languages, which should be followed by the dragon book. The dragon book has a pretty good (but short) section on language design too.

This post has been edited by blackcompe: 20 January 2012 - 10:53 PM

[ishkabible]

steps:

front end and language design:
1) get a loose idea of what you want the language to do(paradigm, language features, etc...)
2) look at syntax for how that would work
3) decide on the semantics(this goes along with step 2)
4) decide on a lexical scheme(not hugely important but can have large impact on how complicated your parser is)
5) write a grammar(important for any language even tcl and lisp or even an assembler; brainfuck might be an exception)
6) check steps 1-5 to make sure everything actually fits together(it won't first go around)
7) write the lexical analyzer
8) write the parser and data structure that stores the parsers output

back end(lots of options)
*you could interpret the output of the parser directly
*you could compile to a intermediate representation and then to a compiled format
*you could compile to a byte-code and interpret that
*you could produce compiled output directly and save it to a file
*you can either run the compiled output using a JITer(one you make or something like GNULightning)
*you can compile to a simpler language like C
*you could use a virtual like LLVM(really a whole back end), CLR, JVM, Parrot

things to note:
*this doesn't include optimization at all
*this is not an exhaustive list of options
*many of these steps are subject change depending on the language at hand(especially languages with meta programing facilities)

edit:
Haskell(maybe ML or OCaml), C/C++(maybe D), Io(maybe Smalltalk), Prolog, Lisp(or Scheme), and some form of assembly would be very helpful to know in designing a language as they cover a wide range of paradigms and styles.

Io and Lisp are very easy to parse(I think prolog is too but I'm not that familiar with it) and have simple(but powerful) semantics.

Io is a purely object oriented language.
Lisp is a functional class of languages with extreme meta programing facilities.
Haskell is a purely functional language with lazy semantics.
Prolog is a declarative logic printed language

O BTW, I say this not because these languages are particularly useful in language devolpment but rather becuase they help with language design.

the lexical analyzer and parser can also be generated using various tools but learning to hand write a parser and lexer is also a good skill.

This post has been edited by ishkabible: 20 January 2012 - 10:45 PM

[dartoscoder]

Thanks for all of your replies. When I get a bit further along with my game programming I might look into this next.

Thanks a lot.

[jon.kiparsky]

This is a pretty interesting book, and you might like to take a look at it. Not a lot of how-to advice, but a lot of talk about the higher-level considerations of language design. I'd say it's worth borrowing from the library, but there's not much in it that's worth owning.

[Shane Hudson]

I am currently myself reading through a library copy of the Dragon Book, it really does seem to be the essential resource for compiler theory (not so much language design yet mind you). I shall have to buy it one day, it looks so much better on my shelf that "Teach yourself C++ in 24 hours" and a couple of other beginners books I have!

[antshockey]

I just finished writing a compiler for my module Languages and Compilers.
We had the language already designed for us but I feel like I could design my own now.
It was only an imperative language and compiled into C not into Assembly or Machine code.
It took a good 100 hours of programming to be fully functional and include code optimisation and this was with a lot of guidance, so one thing I would advise is only start if you're sure it's what you want to do because it will take a long time. (It will also get very frustrating and you will be hunting the most ambiguous errors known to man)
Good luck though haha

[ishkabible]

I actually recommend compiling to C or LLVM code for your first language(or using a virtual machine like parrot). in fact if you were aiming for extreme portability then using C is your best option.

[jon.kiparsky]

ishkabible, on 22 January 2012 - 12:52 PM, said:

I actually recommend compiling to C or LLVM code for your first language(or using a virtual machine like parrot). in fact if you were aiming for extreme portability then using C is your best option.

The accepted best practice seems to be - from my limited study, which might be incorrect - to use a front-end/back-end approach, where you compile to a machine-independant internal representation and then translate that representation to a machine-readable code.
The advantage of this is that you are then getting something for nothing after a certain amount of work is done: when you have written the front end for two languages and the back end for two machines, you now have four compilers (L1->M1, L2->M1, L1->M2, L2->M2), for about the same work that would have produced an L1->M1 and an L2->M2 compiler.
Using C as this internal representation might be very reasonable indeed, since obviously C compilers exist for any machine you're likely to run into, and therefore the back-end side is taken care of already.

[ishkabible]

Quote

...therefore the back-end side is taken care of already.

that's the idea

Compiling to C takes care of basically all your back end needs unless you need more advanced linking options or some specialized feature that only assembly can tackle efficiently. In the case that you need more linking options then LLVM is probably sufficient(passed that your going to need to make your own linker). LLVM also offers necessary facilities to to efficiently compile D and Haskell so it will probably take care of any special features you need.

the other huge benefit is optimization; C compilers offer some of the best(if not the best) optimization of any compiler so you can get a superbly optimized language for no cost!

also some compilers like GHC have many steps; GHC converts first to a core Haskell language which takes away some of the syntactic sugar then to STG(spineless tagless G-machine) then to C-- and then either to LLVM, C, or native code. that's 4 conversions!

if you generate a lexical analyzer and parser then all you have to do is translate that to C and you have a compiler! It can be far less work now days then it used to be.

[macosxnerd101]

*Moved to Computer Science*

You will want to look into natural and formal languages as well.

[jon.kiparsky]

Of course, we should keep in mind that all of this talk about writing compilers has nothing at all to do with the original question, which was about language design. A language need not ever be compiled to be useful (see SPARKS and MIX).

I think the first step would be to decide what problems you want to solve. That's ambiguous, and I mean both "what problems do I want this language to be adapted for solving" and "what problems are there in existing languages that I want to get around in my new language". If you don't start with a problem, it's not likely you'll find a solution to anything in particular.

This post has been edited by jon.kiparsky: 22 January 2012 - 07:33 PM